Creating an Attachment SObject complete with its body using the REST API from Java

Exporting the body of an Attachment using the REST API is fairly straightforward: the “Body” field of the Attachment JSON contains the URL to GET the raw bytes for the Attachment body from. But it took me a while to get the importing of an Attachment and its body to work because PATCH or POST to the equivalent body URL is not allowed.

This documentation Insert or Update Blob Data explains generically what needs to be done. But it took me a bit of time to work out how to do this in Java using the Apache HttpClient. Here is a stripped down version of what I ended up with that might help you if you are writing similar Java code:

package com.claimvantage.ant;

import java.io.File;
import java.io.IOException;
import java.io.OutputStream;

import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.commons.httpclient.methods.multipart.FilePart;
import org.apache.commons.httpclient.methods.multipart.MultipartRequestEntity;
import org.apache.commons.httpclient.methods.multipart.Part;
import org.apache.commons.httpclient.methods.multipart.PartBase;

public class AttachmentExample {
    
    private class JsonPart extends PartBase {
        
        private byte[] bytes;
        
        public JsonPart(String name, String json) throws IOException {
            super(name, "application/json", "UTF-8", null);
            this.bytes = json.getBytes("UTF-8");
        }
        
        @Override
        protected void sendData(OutputStream os) throws IOException {
            os.write(bytes);
        }

        @Override
        protected long lengthOfData() throws IOException {
            return bytes.length;
        }
    }
    
    private String baseUrl;     // Initialization not shown here
    private String sessionId;   // Initialization not shown here
    
    /**
     * Create attachment SObject from its JSON populating its Body from a file at the same time.
     */
    public void createAttachment(String attachmentJson, File attachmentFile) throws Exception {
        
        PostMethod post = new PostMethod(baseUrl + "/services/data/v23.0/sobjects/Attachment");
        post.setRequestHeader("Authorization", "OAuth " + sessionId);
        Part[] parts = new Part[] {
                new JsonPart("Json", attachmentJson),
                new FilePart("Body", attachmentFile)
                };
        post.setRequestEntity(new MultipartRequestEntity(parts, post.getParams()));
        try {
            new HttpClient().executeMethod(post);
            if (post.getStatusCode() == HttpStatus.SC_CREATED) {
                // Logic for OK
            } else {
                // Error handling logic
            }
        } finally {
            post.releaseConnection();
        }
    }
}

“New” action pre-processing override results in “Error: Invalid Data…” on “Save & New”

You can create a custom page and controller and then hook them up to the “New” action for an object – see e.g. Overriding a Standard Button. One situation where this is useful is where you want to do some of your own processing and then forward on to the default UI page.

Here is the sort of page you might use for this purpose:

<apex:page
    standardController="MyCustomObject__c"
    extensions="MyCustomObjectController"
    action="{!url}
    />

and controller:

public with sharing class MyCustomObjectController {

  // Controller extension
  public MyCustomObjectController(ApexPages.StandardController ignored) {
  }

  public PageReference url() {

    // Custom processing logic goes here

    // Standard URL pattern for "New"
    String keyPrefix = MyCustomObject__c.SObjectType.getDescribe().getKeyPrefix();
    PageReference newPr = new PageReference('/' + keyPrefix + '/e');

    // Preserve any query string parameters
    Map<String, String> oldParams = ApexPages.currentPage().getParameters();
    Map<String, String> newParams = newPr.getParameters();
    newParams.putAll(oldParams);

    // Avoid an infinite loop - go to default UI not back to this controller
    newParams.put('nooverride', '1');

    return newPr;
  }
}

This works well when “Save” is clicked in the resulting default UI page. But when “Save & New” is clicked, this (confusing) error message results:

Error: Invalid Data.
Review all error messages below to correct your data.
The page you submitted was invalid for your session. Please click Save again to confirm your change.

After much stripping back of the controller code and comparisons with the typical URL patterns, the problem turns out to be that in the “Save & New” case a save_new parameter is added for part of the cycle but that should not be present in the URL of the final page that is presented.

Adding this line of code (or otherwise making sure the parameter is not present in the final URL) fixes the problem:

    newParams.remove('save_new');

How much harm can one line of code do?

Suppose you are developing a feature that does programmatic manipulation of sharing rules. You go ahead and set the sharing default for your CustomObject to other than “Public Read/Write” in your development org. This automatically creates the CustomObject__Share object to go with your CustomObject__c and so allows you to add a class with a signature such as this:

global class Xyz {
    global void doSomething(List<CustomObject__Share> shares) {
        // ...
    }
}

And then you go ahead and create a managed released package.

The good news is that you now have a managed package that works well for customers who want to configure sharing rules. But the bad news is that you have made sharing rules mandatory for all your customers: the package cannot be installed unless the sharing default for CustomObject is set to other than “Public Read/Write” in the org the package is deployed to.

Woops.

So you go to fix this. The signature you would like to use instead is:

global class Xyz {
    global void doSomething(List<SObject> shares) {
        // ...
    }
}

But then you realize that global method signatures can’t be changed once included in a managed released package. Using @deprecated doesn’t help in this case: it is the presence of CustomObject__Sharing in the signature (whether the method is marked as deprecated or not) that creates the problem. Bottom line is that you are stuck.

Woops WTF.

Be careful what you put in the signature of global methods.

Some lessons learned delivering an enterprise application on Force.com

The app this is based on uses dozens of related custom objects. And pretty much always customers want to add further custom fields and customization logic. The app is also constantly having new features added. These characteristics obviously influence the lessons learned…

  1. Managed released packages work well and so do patch releases. At this point we have created hundreds of versions as we develop incrementally. (Managed beta packages are fairly useless as they don’t support upgrading.) The version compatibility constraints leave junk in the data model and don’t protect against all breaking changes. Customers expect it to be able to back out an upgrade but unfortunately that isn’t supported. There is a major pain point in that the upgrade process does not propagate new versions of things like layouts.
  2. Customer-specific extensions can be delivered using a further managed released package that builds on the core app one or via the Ant deploy task. Given that the former approach does not propagate many common changes on update, the latter approach is usually the best one to use.
  3. Version management just about works (for small teams). The managed packaged org is the definitive version: your Git or SVN copy is your best attempt to track it and doesn’t include information such as the exact set of components in the package. (We use the convention that everything in the org should be in the package.)
  4. The Eclipse based Force.com IDE is poor (as it has had only basic maintenance investment over the last few years) but necessary. It allows the code and other artifacts to be seen and searched as a whole and is the best place to see change and version information.
  5. Unit tests are the safety net that let your managed package change and grow. Use continuous integration server to make sure they are run frequently (and to make sure what you have in Git or SVN is complete).
  6. Governor limit violations mostly but not always result from a design error such as non-linear algorithms (hitting the code statements limit) or non-bulkified queries (hitting the number of SOQL queries limit). But they do come up in other situations. For example, your managed package controller starts some processing that consumes some of its governor limits directly and in its triggers; a trigger added by your customer also runs and updates some of your objects consuming from its own set of governor limits; this then causes triggers back in the managed package to consume more of the managed package governor limits exceeding a limit. Whatever the cause, hitting these limits is a brutal problem: where on other platforms performance might become poor, on Force.com your app is broken and your customer is left unable to work.
  7. There are two eras in how open for extension a Force.com app is. The boundary between these eras is the availability in Summer ’12 of Type.forName and Type.newInstance. These allows extension points to be built into an app and extension code to be added by supplying the names of classes that implement the required interfaces. JSON strings in static resources or directly in custom settings are also a help. Before this it was very hard to add external code.
  8. Embrace the use of Contact and Account. As well as helping your app share this information with other apps, some API calls require the Id value passed to be a Contact Id.
  9. Dynamic SOQL is inelegant (as you lose compile time checks) but necessary in some cases. An example is where you want to do a deep clone of part of your object graph: use describe calls to get all the field names for each object so that custom fields that your managed package does not know about – ones added for specific customers – are included.
  10. While the execution order of before triggers compared to after triggers is defined and important to know and make use of, the execution order of separate triggers for the same event e.g. “before update” on a Contact is not defined. Within one code base discipline and patterns must be applied to define the order. But this problem is hard to address across multiple code bases e.g. your app and extending code triggers.
  11. While it is tempting to write an Apex classes that deals with a single object on the grounds that the code will be cleaner and “it’ll never be used in a bulk situation”, sometime in the future it probably will be used in a bulk situation and result on a governor limit error that stops some important data migration or change from being possible. Always deal in sets of objects.
  12. Client-side JavaScript using e.g. jQuery is a must-have these days. It is easy to introduce into Visualforce pages but has to be hacked into standard layout based UI – see TehNrd’s Show and Hide Buttons on Page Layouts. See Using JavaScript with Force.com for a good overview of other JavaScript techniques.
  13. Sometimes you have the difficult choice between delivering poor usability or instead depending on platform implementation that may change e.g. hack to find field ids.
  14. Profiles are hugely painful to keep updated over time and are rarely tested well enough. Do everything you can to keep the number of them in use small.
  15. On custom settings, and particularly if you are using profile or user-specific hierarchical ones, you will eventually need to use code to keep them updated consistently.

Cleaner inner class test fixture pattern

I favor this inner class test fixture pattern for Apex unit tests where a set of related objects are required. It leverages Apex’s support for named parameters when concrete SObject types are created making the code self describing and quick to modify including when fields are added. (Patterns like builder typically need extra methods adding when new fields are added.) And rather late in the day I’ve noticed that the result of an assignment is the value assigned (as in languages like Java).

These two language features together allow this high signal-to-noise ratio fixture style where a single line initializes, assigns and inserts each object:

@isTest
private class OneTwoThreeTest {
    class Fixture {
        One__c one;
        Two__c two;
        Three__c three;
        Fixture() {
            insert one = new One__c(Number__c = 12.34);
            insert two = new Two__c(One__c = one.Id, String__c = 'Hello');
            insert three = new Three__c(Two__c = two.Id, Checkbox__c = true);
        }
    }
    @isTest
    static void test() {
        Fixture f = new Fixture();
        // ...
    }
}

Ant task that automates the installation of managed packages

At ClaimVantage we deploy using a mixture of managed packages and source code. The development process for a customer can involve up to this many orgs:

  • one (or more) developer edition orgs that developers create the custom code in
  • a build org that the Jenkins continuous integration server uses to verify that the code committed to the version control system is deployable including 100% passing tests
  • a QA sandbox org that the customer’s QA team test in
  • a UAT sandbox org that the customer’s business users test in
  • the customer’s production org

And as we typically work in 2 week long iterations, over a few months of work for a customer many deployments get done to each of these orgs.

The source code can be pushed into orgs using the Ant tools provided by Salesforce. While it is not quick – typically it takes a few minutes – it requires no attention once started so you can get on with other work and check the outcome later.

But managed package installation unfortunately involves a multiple-page wizard with slow transitions between the pages even before the final asynchronous step. And Salesforce have not provided an API or any other tooling to allow this tedious button clicking to be eliminated.

So I am very grateful that my colleague Sinead Coyle has created an Ant task that automates the clicking via Selenium. She is now sharing this in the Google code project force-managed-package-installer. It allows the managed package installation or upgrade to be just another step in the Ant script ahead of the source code deployment.

FullCalendar using a JavaScript remoting callback

FullCalendar quick start illustrates a simple way to get FullCalendar working in a Visualforce page. All the event data was just emitted as JSON text in the generated page. But FullCalendar is typically used where the date range being displayed can be changed making that approach impractical. Two alternate ways to supply the event data are supported:

  • via a json feed – a GET url is configured
  • via a function – an arbitrary piece of JavaScript is configured

Both of these are passed the date range that is required for the calendar so that only the relevant events can be returned. The second of these provides more opportunity to do processing on the client side and is the approach described below.

Here is a simplified version of the Apex controller. The Event class models the values required for each event by FullCalendar and Apex automatically serializes the returned list of these as JSON because of the RemoteAction annotation. The arguments to the events method are two Apex Date primitives and an SObject (whose fields are displayed in the page and provide additional control over the calendar content):

public with sharing class CalendarController {
    
    public class Event {
    
        public String title;
        public String tip;
        public DateTime starts;
        public DateTime ends;
        
        public Event(String title, String tip, DateTime starts, DateTime ends) {
            this.title = title;
            this.tip = tip;
            this.starts = starts;
            this.ends = ends;
        }
    }

    // Page calls back into this
    @RemoteAction
    public static Event[] events(Date starts, Date ends, Filter__c filter) {

        System.debug('>>> starts=' + starts + ' ends=' + ends + ' filter=' + filter);

        List<Event> events = new List<Event>();

        // Loop goes here here where query results are turned into Event objects

        return events;
    }
}

Here is the change to what is assigned to “events” in the Visualforce page. Each time FullCalendar requires event data it invokes this JavaScript function. The function obtains values from the SObject fields in the page and passes those together with the start and end dates to the controller via the standard remote action call that automatically serializes the request as JSON. The returned results are passed into FullCalendar by invoking the callback function supplied in the original function call:

events: function(start, end, callback) {
    
    // JQuery references to the hidden id fields for lookups
    // and also to checkbox fields assigned to fields whose
    // names match the Filter__c SObject
    var filter = {
        Booking__c: bookingLkid.val(),
        Class__c: clazzLkid.val(),
        ShowAvailable__c: showAvailable.prop('checked'),
        ShowBooked__c: showBooked.prop('checked')
    };
    
    // JavaScript dates need formatting to be accepted at the Apex side
    Visualforce.remoting.Manager.invokeAction(
        '{!$RemoteAction.CalendarController.events}',
        start.toUTCString(),
        end.toUTCString(),
        filter, 
        function(result, event) {
            if (event.status) {
                // Jobs done here:
                // 1) In Apex "end" is a reserved word so using "ends"; change back
                // 2) The Apex DateTime is serialized as milliseconds; change to Date
                // 3) The HTML < and > are replaced by entities; turn back
                for (var i = 0; i < result.length; i++) {
                    var r = result[i];
                    r.start = new Date(r.starts);
                    r.end = new Date(r.ends);
                    r.tip = r.tip.replace(/&lt\;/g, '<').replace(/&gt\;/g, '>');
                }
                // Invoke FullCalendar function
                callback(result);
            } else {
                alert('ERROR:\n' + event.message + '\n' + event.where);
            }
        });
}

Gotchas along the way were (API 25.0):

  • Although the remote class and method being invoked are explicit in the call, if the class isn’t the controller or an extension, a page build error of “No remoted actions found to resolve…” results.
  • The automatic serialization of Date appears broken in both directions: see the JavaScript above for work-arounds; there may be similar issues with other types.
  • As the JSON serialization can’t be configured its awkward to generate JSON that includes any Apex reserved words.
  • The “tip” string is HTML generated by the Apex code and so needed some manipulation on the client-side to remove the encoding introduced by the serialization.
  • The Force.com documentation is short on details and examples so expect to spend time debugging to figure out what is actually happening.

Finally I’d like to mention that adding a tooltip to FullCalendar took less than 5 minutes and worked first time: all that was needed was the qTip JavaScript and this small addition:

eventRender: function(event, element) {
    element.qtip({
        content: event.tip
    });
}

Only some types work as Apex map keys from Visualforce

The “Referencing Apex Maps and Lists” section of the Visualforce Developer’s Guide provides examples of how to reference data held in an Apex map in the controller from a Visualforce page. I haven’t found examples where the key values used have been other than strings or integers and the documentation is not explicit. But now that there is support for Non-primitive Types in Map Keys and Sets, I thought it would be worth trying a DateTime value as a map key in a page I have that heavily uses DateTime objects.

But DateTime as a map key type does not work from Visualforce:

Incorrect parameter type for subscript. Expected java.lang.Class, received DateTime

To get some sense of what does and what does not work I created the controller and page listed at the end of this post (both set to API 26.0). I found two further failing map key types.

Time as a map key type results in:

Unsupported type shared.xml.soap.Time encountered.

A custom Apex class as a map key type results in:

Map key Custom:[value=987] not found in map

i.e. value semantics are not supported (but the generally less useful reference semantics are).

I’ll create a case with Salesforce about the types that don’t work.

<apex:page controller="MapTestController">
  <h1>Map Test</h1>
  <br/><apex:outputText value="{!stringKeyMap[stringKey]}"/>
  <br/><apex:outputText value="{!integerKeyMap[integerKey]}"/>
  <br/><apex:outputText value="{!longKeyMap[longKey]}"/>
  <br/><apex:outputText value="{!decimalKeyMap[decimalKey]}"/>
  <br/><apex:outputText value="{!dateKeyMap[dateKey]}"/>
  <br/><apex:outputText value="{!customByReferenceKeyMap[customByReferenceKey]}"/>
  <!--
  <br/><apex:outputText value="{!timeKeyMap[timeKey]}"/>
  -->
  <!--
  <br/><apex:outputText value="{!dateTimeKeyMap[dateTimeKey]}"/>
  -->
  <!--
  <br/><apex:outputText value="{!customByValueKeyMap[customByValueKey]}"/>
  -->
</apex:page>
public class MapTestController {

    public String stringKey {
        get {
            return 'key';
        }
    }
    public Map<String, String> stringKeyMap {
        get {
            return new Map<String, String>{stringKey => 'string value'};
        }
    }
    
    public Integer integerKey {
        get {
            return 1234;
        }
    }
    public Map<Integer, String> integerKeyMap {
        get {
            return new Map<Integer, String>{integerKey => 'integer value'};
        }
    }
    
    public Long longKey {
        get {
            return 1234;
        }
    }
    public Map<Long, String> longKeyMap {
        get {
            return new Map<Long, String>{longKey => 'long value'};
        }
    }
    
    public Decimal decimalKey {
        get {
            return 3.25;
        }
    }
    public Map<Decimal, String> decimalKeyMap {
        get {
            return new Map<Decimal, String>{decimalKey => 'decimal value'};
        }
    }
    
    public Date dateKey {
        get {
            return Date.newInstance(2012, 1, 1);
        }
    }
    public Map<Date, String> dateKeyMap {
        get {
            return new Map<Date, String>{dateKey => 'date value'};
        }
    }
    
    public Time timeKey {
        get {
            return Time.newInstance(8, 20, 30, 0);
        }
    }
    public Map<Time, String> timeKeyMap {
        get {
            return new Map<Time, String>{timeKey => 'time value'};
        }
    }
    
    public DateTime dateTimeKey {
        get {
            return DateTime.newInstance(2012, 1, 1, 0, 0, 0);
        }
    }
    public Map<DateTime, String> dateTimeKeyMap {
        get {
            return new Map<DateTime, String>{dateTimeKey => 'date time value'};
        }
    }
    
    public class Custom {
        private Integer value;
        public Custom(Integer value) {
            this.value = value;
        }
        public Boolean equals(Object other) {
            return this.value == ((Custom) other).value;
        }
        public Integer hashCode() {
            return value;
        }
    }
    
    private static final Custom customInstance = new Custom(321);
    public Custom customByReferenceKey {
        get {
            return customInstance;
        }
    }
    public Map<Custom, String> customByReferenceKeyMap {
        get {
            return new Map<Custom, String>{customByReferenceKey => 'custom (by reference) value'};
        }
    }
    
    public Custom customByValueKey {
        get {
            return new Custom(987);
        }
    }
    public Map<Custom, String> customByValueKeyMap {
        get {
            return new Map<Custom, String>{customByValueKey => 'custom (by value) value'};
        }
    }
}

FullCalendar quick start

I was pleasantly surprised at how painlessly Adam Shaw’s FullCalendar worked in a Visualforce page so am posting a basic controller and page here to help others get started. This is the output they produce (with of course all the hard work done by FullCalendar):

The controller emits hard-coded data: a real controller would build the data from various SObjects and provide actions to be invoked from the client-side JavaScript:

public with sharing class CalendarDemoController {
    public String events {
        get {
            JSONGenerator g = JSON.createGenerator(true);
            g.writeStartArray();
            for (Integer i = -2; i <= 2; i++) {
                g.writeStartObject();
                g.writeStringField('title', 'Event ' + i);
                g.writeDateTimeField('start', DateTime.now().addDays(i).addHours(i));
                g.writeDateTimeField('end', DateTime.now().addDays(i).addHours(i + 1));
                g.writeEndObject();
            }
            g.writeEndArray();
            return g.getAsString();
        }
    }
}

The fullcalendar-1.5.4.zip download is setup as a single static resource with the various pieces of content accessed via their paths in the Visualforce page. The calendar is rendered using a mixture of static configuration and the events JSON string provided by the controller:

<apex:page controller="CalendarDemoController" tabStyle="Contact">

<link href="{!URLFOR($Resource.fullcalendar,'fullcalendar-1.5.4/fullcalendar/fullcalendar.css')}" rel="stylesheet" type="text/css"/>
<link href="{!URLFOR($Resource.fullcalendar,'fullcalendar-1.5.4/fullcalendar/fullcalendar.print.css')}" rel="stylesheet" type="text/css" media="print"/>
<script src="{!URLFOR($Resource.fullcalendar, 'fullcalendar-1.5.4/jquery/jquery-1.8.1.min.js')}" type="text/javascript" language="javascript"></script>
<script src="{!URLFOR($Resource.fullcalendar, 'fullcalendar-1.5.4/fullcalendar/fullcalendar.min.js')}" type="text/javascript" language="javascript"></script>

<apex:sectionHeader title="Calendar Demo"/>

<!-- Calendar is rendered here by the fullcalendar Javascript -->
<apex:pageBlock >
    <div id='calendar'/>
</apex:pageBlock>

<!-- Fullcalendar configuration plus event data supplied by controller -->
<script type="text/javascript" language="javascript">
$(document).ready(function() {
    $('#calendar').fullCalendar({
        header: {
            left: 'title',
            center: 'month,agendaWeek,agendaDay',
            right: 'prevYear prev,today,next nextYear'
        },
        defaultView: 'agendaWeek',
        allDayDefault: false,
        minTime: 8,
        maxTime: 21,
        events:
{!events}
    });
});
</script>
</apex:page>