Fixing a common cause of System.LimitException: Apex CPU time limit exceeded

When developing code, automated unit tests and interactive testing naturally tend to use small numbers of objects. As the number of objects increases, the execution time has to increase in proportion – linearly. But it is all too easy to introduce code where the execution time grows as the square of the number of objects or the cube of the number of objects. (See e.g. Time complexity for some background.) For example, even with 100 objects, code that takes 100ms for the linear case, takes 10s for the squared case and 1,000s for the cubed case. So for the squared or cubed cases

System.LimitException: Apex CPU time limit exceeded

exceptions can result with even modest numbers of objects.

Is this governor limit a good thing? On the positive side, it forces a bad algorithm to be replaced by a better one. But on the negative side, your customer is stuck unable to work until you can deliver a fix to them. Some sort of alarm and forgiveness from the platform for a while would be more helpful…

Here is a blatant example of the problem (assuming all the collections include large numbers of objects):

// Linear - OK
for (Parent__c p : parents) {
    // Squared - problem
    for (Child__c c : children) {
        // Cubed - big problem
        for (GrandChild__c bc : grandChildren) {
            // ...
        }
    }
}

So review all nested loops carefully. Sometimes the problem is hidden, with one loop in one method or class and another loop in another method or class, and so harder to find.

Often the purpose of the loops is just to find the objects/object in one collection that match an object in another collection. There are two contexts that require two different approaches (though in more complicated cases the approaches can be combined) to fix the problem or better to avoid it in the first place:

  • In e.g. a controller where the query is explicit, a parent and child can be queried together with direct relationship references available using a relationship query.
  • In a trigger, no __r collections are populated, so maps have to be used. A map allows a value to be looked up without the cost of going through every entry in a list.

Here is how to fix the problem for the two contexts in parent-child relationships:

// Explicit SOQL context
for (Parent__c p : [
        select Name, (select Name from Childs__r)
        from Parent__c
        ]) {
    // Loop is over the small number of related Child__c not all of the Child__c
    for (Child__c c : p.Childs__r) {
        // ...
    }
}


// Trigger context
Map<Id, List<Child__c>> children = new Map<Id, List<Child__c>>();
for (Child__c c : [
        select Parent__c, Name
        from Child__c
        where Parent__c in Trigger.newMap.keySet()
        ]) {
    List<Child__c> l = children.get(c.Parent__c);
    if (l == null) {
        l = new List<Child__c>();
        children.put(c.Parent__c, l);
    }
    l.add(c);
}
for (Parent__c p : Trigger.new) {
    // Loop is over the small number of related Child__c not all of the Child__c
    if (children.containsKey(p.Id)) {
        for (Child__c c : children.get(p.Id) {
            // ...
        }
    }
}

And for the two contexts in child-parent relationships:

// Explicit SOQL context
for (Child__c c : [
        select Name, Parent__r.Name
        from Child__c
        ]) {
    // The one parent
    Parent__c p = c.Parent__r;
    if (p != null) {
        // ...
    }
}


// Trigger context
Set<Id> parentIds = new Set<Id>();
for (Child__c c : Trigger.new) {
    if (c.Parent__c != null) {
        parentIds.add(c.Parent__c);
    }
}
if (parentIds.size() > 0) {
    Map<Id, Parent__c> parents = new Map<Id, Parent__c>([
            select Id, Name
            from Parent__c
            where Id in :parentIds
            ]);
    for (Child__c c : Trigger.new) {
        // The one parent
        if (c.Parent__c != null) {
            Parent__c p = parents.get(c.Parent__c);
            if (p != null) {
                // ...
            }
        }
    }
}

This fixed code will operate in close to linear time.

Inner class test fixture pattern

Most Force.com tests require one or more related SObjects to be created in a known configuration. There are many ways this can be coded:

  • Inline in the static test method: but offers no re-use
  • Created by a static helper method in the test class: but not easy for the test method to access more than one of the created objects
  • Created by a separate fixture class: but hard to support all the variations needed and a lack of cohesion

In general I suggest the pattern illustrated below – using an inner test fixture class – is more effective. It addresses the problems listed above and is tolerant of change:

@isTest
private class XyzTest {
    // If variations in the Fixture are needed an enum is a clean way of identifying them
    private enum TestCase {
        Case1,
        Case2
    }
    private class Fixture {
        public Parent__c parent;
        public Child__c[] children;
        public Fixture(TestCase tc) {
            parent = new Parent__c(FirstName__c = 'Jane', LastName__c = 'Doe');
            insert parent;
            children = new Child__c[] {
                    new Child__c(Parent__c = parent.Id, Gender__c = 'Female'),
                    new Child__c(Parent__c = parent.Id, Gender__c = 'Male')
                    };
            insert children;
        }
        public void assertParent() {
            Parent actual = [select ... from Parent__c where Id = :parent.Id];
            // System.asserts go here
        }
        public void assertChildren() {
            Child__c[] actual = [select ... from Child__c where Id in :SobUtil.getIds(children)];
            // System.asserts go here
        }
    }
    @isTest
    static void testCase1() {
        Fixture fixture = new Fixture(TestCase.Case1);
        // Code here can access fields, perform assertions and invoke the fixture's assertions
    }
    @isTest
    static void testCase2() {
        Fixture fixture = new Fixture(TestCase.Case2);
        // Code here can access fields, perform assertions and invoke the fixture's assertions
    }
}

Accessing record type IDs in Apex code

Where you use record types on several objects, a convenient way to obtain the record type ID in Apex is needed. And as the record type ID requires a SOQL query to obtain the org-specific value, governor limits and efficiency come in to play too.

The pattern illustrated below allows the ID to be obtained using this simple and flexible call:

Id id = RT.getId(cve__Claim__c.SObjectType, RT.Name.ShortTermDisability);

The benefits of this approach are:

  • Only one SOQL query done (per server request) no matter how many record type IDs are referenced or where they are referenced e.g. inside a loop.
  • Avoids problems of typos in strings by using the compile-time checked SObjectType field and an enum for the Name values.
  • Handles the case where multiple objects use the same record type name well.
  • Adding a new record type just requires a single line to be added to the Name enum.
/**
 * RT stands for "RecordType".
 * Named cryptically as typical usage includes this class name twice.
 */
public class RT {
    /**
     * These (developer) names can apply to more than one SOBjectType.
     */
    public enum Name {
        ShortTermDisability,
        LongTermDisability,
        TermLife,
        WaiverOfPremium
    }
    private static final String SEPARATOR = '::::';
    private static Map<String, Id> CACHE;
    /**
     * Get the Id that can be set on or compared with an SOBject's RecordTypeId field.
     * Backed by a cache of all record type ids for the duration of a request.
     */
    public static Id getId(SObjectType sobType, Name developerName) {
        if (CACHE == null) {
            CACHE = new Map<String, Id>();
            for (RecordType rt :  [
	                select Id, SObjectType, DeveloperName
	                from RecordType
	                where DeveloperName in :getNames()
	                ]) {
                CACHE.put(rt.SObjectType + SEPARATOR + rt.DeveloperName, rt.Id);
            }
        }
        return CACHE.get(String.valueOf(sobType) + SEPARATOR + developerName.name());
    }
    private static Set<String> getNames() {
    	Set<String> names = new Set<String>();
    	for (Name name : Name.values()) {
    		names.add(name.name());
    	}
    	return names;
    }
}

Also check out GaryB’s Record Types approach.

Automatically setting the record type of a detail object based on the record type of its master object

Record types are defined on individual objects and when an object that has a record type is created using the default UI the user is first presented with a page where the record type is selected and then is presented with a normal edit page controlled by the selected record type. But if an object model has master and detail objects where the record types need to be consistent, this manual setting is inappropriate for the detail object because it allows an inconsistent value to be set and also because it forces the user to fill in an extra, redundant page.

This post describes a technique for skipping the manual setting of the record type for the detail object and instead automatically setting it. An important part of this is the “Skip Record Type Selection Page” option accessed via an object’s “Standard Buttons and Links” override for “New”:

The normal “New” page is replaced by the specified page that must be based on the object’s standard controller (to be offered in the override page) and add its logic via a controller extension:

<apex:page
        standardController="PaymentSpecification__c"
        extensions="PaymentSpecificationNewController"
        action="{! url }"
        />

The page is never rendered but instead invokes its controller’s url method that returns the URL that is rendered.

The controller is implemented in two classes to separate the core logic that applies to any master/detail pair from the specific master/detail information. Here BenefitClaimed is the master object and PaymentSpecification is the detail object (created using the “New” button above the related list of PaymentSpecification objects in the BenefitClaimed page):

public with sharing class PaymentSpecificationNewController extends NewChildControllerBase {
    public PaymentSpecificationNewController(ApexPages.StandardController ignored) {
        super(BenefitClaimed__c.SObjectType, PaymentSpecification__c.SObjectType);
    }
}

The base class contains the core logic:

public abstract class NewChildControllerBase {
    private SObjectType parentSobType;
    private SObjectType childSobType;
    public NewChildControllerBase(SObjectType parentSobType, SObjectType childSobType) {
        this.parentSobType = parentSobType;
        this.childSobType = childSobType;
    }
    public PageReference url() {
        PageReference p = new PageReference('/' + childSobType.getDescribe().getKeyPrefix() + '/e');
        Map<String, String> m = p.getParameters();
        m.putAll(ApexPages.currentPage().getParameters());
        m.put('RecordType', getChildRecordTypeId(getParentSObjectId(m)));
        m.put('nooverride', '1');
        return p;
    }
    private Id getChildRecordTypeId(Id parentId) {
        SObject parent = Database.query('select RecordType.DeveloperName from ' + String.valueOf(parentSobType)
                + ' where Id = :parentId');
        String parentDeveloperName = (String) parent.getSobject('RecordType').get('DeveloperName');
        return [select Id from RecordType where SObjectType = :String.valueOf(childSobType)
                and DeveloperName = :parentDeveloperName].Id;
    }
    private Id getParentSObjectId(Map<String, String> m) {
        for (String key : m.keySet()) {
            if (key.endsWith('_lkid')) {
                return m.get(key);
            }
        }
        return null;
    }
}

The approach used is to take all the parameters from the original request, then add the parameters RecordType and nooverride, and then use those parameters with the normal object edit URL. Keeping the original request parameters is important because they include the values needed to pre-populate the parent object name and id in the child object (see Lkid hack value is fragile for some backround information on this; the approach used here is not fragile) and the URL to return to on cancel. The RecordType parameter supplies the detail object’s record type id and the nooverride parameter stops an infinite loop by ensuring the request is handled by the default UI page rather than the override page.

The record type id for the detail object is obtained by using the id of the master object, obtained from the “_lkid” suffixed parameter, to query the master object’s record type. This is then used to obtain the record type id for the detail object. In this case both record types have the same DeveloperName so that is used in the matching logic but other mappings can obviously be implemented. The dynamic SOQL is used to keep this code generic and configured by just the two SObjectType constructor parameters.

An object Id in Apex that matches no objects

SOQL and SOSL both support the “in” operator in their “where” clauses. I was recently using this mechanism where a small number of Id values were being generated in some code and passed about as Set<Id>. But in one case I wanted the set to contain only an Id that would not match any of the objects. Where this is done directly in SOQL or SOQL, I have seen the string ‘0000000000000000’ (15 zeroes) used and so assumed that would be ok. But when such a String is cast to an Id you get an exception that contains the message “Invalid id”. It appears that the Apex code that handles Ids is a little fussier than the SOQL and SOSL engines.

The good news though is that there appears to be a simple solution to the problem as illustrated in the testOk method of the code below. The getKeyPrefix method “Returns the three-character prefix code for the object” and this combined with 12 zeros can be converted to an Id and appears to never match any objects. But it does cost an expensive and governor limited describe call per object type.

@isTest
private class IdTest {
    @isTest
    static void testOk() {
        ok(Account.SObjectType.getDescribe().getKeyPrefix() + '000000000000', Account.SObjectType);
        ok(Contact.SObjectType.getDescribe().getKeyPrefix() + '000000000000', Contact.SObjectType);
        ok(Task.SObjectType.getDescribe().getKeyPrefix() + '000000000000', Task.SObjectType);
    }
    @isTest
    static void testNotOk() {
        notOk('000000000000000', Account.SObjectType);
        notOk('123456789012345', Account.SObjectType);
        notOk(Account.SObjectType.getDescribe().getKeyPrefix() + '123456789012', Account.SObjectType);
        notOk(Account.SObjectType.getDescribe().getKeyPrefix() + '222222222222', Account.SObjectType);
    }
    private static void ok(String idString, SObjectType sobType) {
        Id id = (Id) idString;
        System.assertEquals(0, Database.countQuery('select Count() from ' + sobType + ' where Id = \'' + id + '\''));
    }
    private static void notOk(String idString, SObjectType sobType) {
        try {
            Id id = (Id) idString;
            System.assert(false, 'exception expected');
        } catch (Exception e) {
            System.debug('exception=' + e);
            System.assert(e.getMessage().contains('Invalid id'));
        }
        // While the above cast fails, the string value can be use in SOQL
        System.assertEquals(0, Database.countQuery('select Count() from ' + sobType + ' where Id = \'' + idString + '\''));
    }
}

A utility that has utility

Most code bases contain utility classes that get used when written but over time get forgotten about because their scope is too narrow. But a utility I keep coming back to in my code is this:

public class SobUtil {
    // Get the id values from a list of objects
    public static List<Id> getIds(List<SObject> sobs) {
        List<Id> ids = new List<Id>();
        for (SObject sob : sobs) {
            ids.add(sob.Id);
        }
        return ids;
    }
    // Get the id values from a specific field in a list of objects
    public static List<Id> getIds(List<SObject> sobs, SObjectField field) {
        List<Id> ids = new List<Id>();
        for (SObject sob : sobs) {
            Id theId = (Id) sob.get(field);
            if (theId != null) {
                ids.add(theId);
            }
        }
        return ids;
    }
}

There is nothing complicated here. The value is that frequently Apex code is dealing with lists of objects not single objects to keep the number of SOQL calls down. So situations where a list of id values need to be extracted are common and the utility class makes such code cleaner.

For example:

// Read the child objects for a collection of parent objects
Parent__c[] parents = ...;
Child__c[] children = [select ChildField1__c, ChildField2__c, ... from Child__c where Parent__c in :SobUtil.getIds(parents)];

Or the reverse relationship (note the expression “Child__c.Parent__c” evaluates to the field metadata type SObjectField rather than a data value; this technique provides compile-time checking that using strings to identify fields does not):

// Read the parent objects for a collection of child objects
Child__c[] children = ...;
Parent__c[] parents = [select ParentField1__c, ParentField2__c, ... from Parent__c where Id in :SobUtil.getIds(children, Child__c.Parent__c)];

Or a common situation in unit tests where to assert the changes made by a trigger the objects have to be re-read:

// Re-read a set of objects
Object__c[] objects = ...;
Object__c[] actuals = [select ObjectField1__c, ObjectField2__c, ... from Object__c where Id in :SobUtil.getIds(objects)];

But remember that where graphs of objects are involved the first port of call is SOQL relationship queries. Connecting parents and children together programmatically only makes sense in certain circumstances such as in a wizard where the list of objects selected in one page are used to query the list of objects presented in a later page.

SOQL and Apex patterns for multiple related objects

Business logic for one object often requires fields in related objects to be accessed. Suppose you have a number of objects related like this:

(In reality each object would have a number of fields that need to be accessed; in the examples below only the “Name” field is being accessed. Also see the earlier posting A SOQL relationship query example for an introductory example and some important links.)

With a Java background, your first thought on how to write the data loading part of such business logic code might be something like this where each object is lazy loaded as required through a getter method:

public class ClaimLogic {
    private Id claimId;
    private Group__c group;
    private Policy__c policy;
    private Claim__c claim;
    private List<Benefit__c> benefits;
    private List<Relationship__c> relationships;
    private List<Contact> contacts;
    public ClaimLogic(Id claimId) {
        this.claimId = claimId;
    }
    private Claim__c getClaim() {
        if (claim == null) {
            claim = [select Name from Claim__c where Id = :claimId];
        }
        return claim;
    }
    private Policy__c getPolicy() {
    	if (policy == null) {
            policy = [select Name from Policy__c where Id = :getClaim().Policy__c];
    	}
    	return policy;
    }
    // 4 other lazy-loading methods
}

But the pattern I suggest considering instead loads all the data at once:

public class ClaimLogic {
    private Group__c group;
    private Policy__c policy;
    private List<Benefit__c> benefits;
    private List<Relationship__c> relationships;
    private List<Contact> contacts;
    public ClaimLogic(Id claimId) {
        claim = [
            select
                    Policy__r.Group__r.Name,                // Grand parent field
                    Policy__r.Name,                         // Parent field
                    Name,                                   // Field
                    (
                            select
                                    Name                    // Child field
                            from Benefits__r
                            where Name like 'A%'            // Child field filtering
                    ),
                    (
                            select
                                    Name,                   // Association object field
                                    Contact__r.Name         // Many to many associated object field
                            from Relationships__r
                            where Contact__r.Name like 'X%' // Many to many associated object field filtering
                    )
            from Claim__c
            where Id = :claimId
            ];
        group = claim.Policy__r.Group__r;
        policy = claim.Policy__r;
        benefits = claim.Benefits__r;
        relationships = claim.Relationships__r;
        contacts = new List<Relationship__c>();
        for (Relationship__c relationship : claim.Relationships__r) {
            contacts.add(relationship.Contact__r);
        }
    }
}

The advantages of this approach are:

  • Instead of six SOQL queries being executed only one SOQL query is executed. Each query takes a few milliseconds and more importantly there is a governor limit on how many in total can be executed per request so six times fewer is a good thing for both performance and avoiding hitting the brick-wall of the governor limit.
  • While not taken advantage of in the above code, by querying all the objects in one go, the relationship fields (the fields ending in “__r”) in both directions are populated. This allows the whole object graph to be passed around and the various parts of the graph accessed as needed in various methods.

It is when writing this kind of SOQL that clear naming of the relationships (done when the relationships are created) becomes important. The parent relationship name is always taken from the parent object name. The child relationship name can be edited when the relationship is created and it is worth taking care with this to make it clear (and plural). The diagram at the beginning of this post shows the relationship names used in the SOQL in bold.

A few more comments on the SOQL:

  • As well as parent objects, grand-parent objects (and presumably great grand-parent objects – not sure if there is a limit) can be accessed.
  • On child objects, only the immediate children of the object in the root query can be accessed i.e. the grand-children cannot be accessed. So it is important to think about which object to use in the root query; sometimes moving up or down a level results in a solution. The relationship between the root query object and the child object is implicit which takes a bit of getting used to if you have a SQL background. The brackets around the child query are required and it can sometimes take a few goes to get those and the relationship name right.
  • The ability to query objects that are in a many to many relationship with the root object via an association object is a nice surprise.
  • Fields referenced via parent relationships can be used in “where” and “order by” clauses too.