Avoiding “too many concurrent batch jobs” in an InstallHandler

A managed package application I work on has a data model that makes it necessary to calculate many values via triggers rather than via SObject formulas in response to specific data changes. The application has been developed incrementally meaning that the fields and logic were introduced in various versions. So InstallHandler code is needed to populate those fields when an upgrade is done.

The original approach to this was to make a series of calls to Database.executeBatch with a Database.Batchable passed in to each one which worked fine until there were 6 of these calls. Then when upgrading an early version (or running the install handler’s tests) this governor limit came in to play:

System.LimitException: Attempted to schedule too many concurrent batch jobs in this org (limit is 5).

So how to get the many benefits of using batch Apex while limiting the number of batch jobs created? The approach shown below makes use of batch chaining where in the finish method of one batchable kicks off the next batchable. To allow the existing batchables to be re-used without change, they are wrapped in a ChainedJob class. This class holds the reference to the next ChainedJob, and delegates to the batchable it wraps; it also holds the scope size value used to execute the batchable.

The install handler can then be coded like this where zero or more ChainedJobs are created and then executed in a chain:

public class XyzInstallHandler implements InstallHandler {

    private Version previousVersion;
    
    public void onInstall(InstallContext context) {
        
        previousVersion = context.previousVersion();
        
        List<ChainedJob> jobs = new List<ChainedJob>();
        if (isInstalledVersionBefore(new Version(2, 12))) {
            jobs.add(new ChainedJob(new ContactBatchable(), 200, 'contact fields'));
        }
        if (isInstalledVersionBefore(new Version(2, 19))) {
            jobs.add(new ChainedJob(new AccountBatchable(), 1000, 'account fields'));
        }
        if (isInstalledVersionBefore(new Version(2, 44))) {
            jobs.add(new ChainedJob(new BenefitBatchable(), 200, 'benefit fields'));
            jobs.add(new ChainedJob(new PolicyBatchable(), 200, 'policy fields'));
        }
        if (isInstalledVersionBefore(new Version(3, 2))) {
            jobs.add(new ChainedJob(new PaidBatchable(), 100, 'paid fields'));
        }
        if (isInstalledVersionBefore(new Version(3, 29))) {
            jobs.add(new ChainedJob(new IncurredBatchable(), 100, 'incurred fields'));
        }
        ChainedJob.executeAsChain(jobs);
    }
    
    private Boolean isInstalledVersionBefore(Version version) {
        return previousVersion != null && previousVersion.compareTo(version) < 0;
    }
}

The ChainedJob class implements Database.Stateful so that the values of its members are maintained through the series of transactions. Note that the class is hard coded to expect the batchable start method to return a Database.QueryLocator (which is the pattern that offers the highest governor limit capability).

public class ChainedJob implements Database.Batchable<SObject>, Database.Stateful {
    
    public String description {get; private set;}
    
    private Database.Batchable<SObject> delegate;
    private Integer scope;
    private ChainedJob nextJob;
    
    public ChainedJob(
            Database.Batchable<SObject> delegate,
            Integer scope,
            String description
            ) {
        this.delegate = delegate;
        this.scope = scope;
        this.description = description;
    }
    
    public Database.QueryLocator start(Database.BatchableContext context) {
        return (Database.QueryLocator) delegate.start(context);
    }
    
    public void execute(Database.BatchableContext context, List<SObject> scope) {
        delegate.execute(context, scope);
    }
    
    public void finish(Database.BatchableContext context) {
        try {
            delegate.finish(context);
        } finally {
            if (nextJob != null) {
                // Next job
                nextJob.execute();
            }
        }
    }
    
    // Create the list and then invoke this method
    public static Id executeAsChain(List<ChainedJob> jobs) {
        if (jobs.size() > 0) {
            for (Integer i = 0; i < jobs.size() - 1; i++) {
                jobs[i].nextJob = jobs[i + 1];
            }
            // First job
            return jobs[0].execute();
        } else {
            return null;
        }
    }
    
    private Id execute() {
        System.debug('>>> executing "' + this.description + '"');
        return Database.executeBatch(this, scope);
    }
}
Advertisements

Winter ’14 – Maximum CPU time woes

From an org that has multiple managed packages installed plus additional source code:

Number of code statements: 148143 out of 200000 ******* CLOSE TO LIMIT
Maximum CPU time: 37266 out of 10000 ******* CLOSE TO LIMIT

Looks like a situation where counting the governor limit across all namespaces rather than per namespace really hurts. See e.g. Script Limits, Begone for the background to this.

2000 metadata JDBC driver downloads

OK so downloading isn’t the same as actively using, but nevertheless I’m pleased that my force-metadata-jdbc-driver JAR that allows SchemaSpy to produce ERD output like the sample below for a Force.com org has been downloaded 1000 times (as of September 2013) 2000 times (as of September 2015).

sample-schemaspy-page