Batch#

Writing one#

public class PostCleanupBatch extends AbstractBatch {

	@Override
	public String batchName () { return "記事の掃除"; }

	@Override
	public String cron () {

		// 毎日 3:15
		return "15 3 * * *";

	}

	@Override
	public void execute (BatchArgs args) {

		int days = settings().getInt("days");

		while (!isCancelOrder()) {
			// 長い処理は中断指示を見る（要件 F-B-06）
		}

	}

}

• batchName() is the name that shows up on the admin screen
• cron() is a five-field cron expression. Write one and the scheduler picks the batch up (isEnableScheduler() is true by default)
• Do not make isScheduler() return true. It asks whether the batch is the scheduler itself; return true and the batch never gets a master row, so cron never runs it
• settings() is the configuration held in the DB (defaultBatchSettings() supplies the initial values)

Registering it#

Nothing is scanned, so you register it yourself.

BatchRegistry.add(PostCleanupBatch::new);

// Bring the list in the DB in line with what is registered right now
BatchRegistry.sync(DBUtil.getMainDB());

sync sets status = nothing on the rows for batches that are gone from the code (the row stays, so the history is still reachable).

[!WARNING] Register everything before you call sync. "Nothing is registered" is treated as "everything is gone," so calling it before you register will set every row to nothing (and the scheduler then runs nothing at all). An app with no batches should not call it.

Do not share batch_master between apps. Whether a batch is gone is decided by class name, so two apps looking at the same table will set each other's rows to nothing.

Making it cancellable#

while (!isCancelOrder()) {
	// handle one at a time
}

Press cancel on the admin screen and isCancelOrder() turns true. In anything long-running, always check it. A batch that never looks cannot be stopped. Always check it in anything long-running. A batch that never looks cannot be stopped.

From code that does not extend AbstractBatch (a domain-layer call, say), the same signal is reachable through the context.

BatchContext context = Context.current(BatchContext.class);

for (Data row : rows) {
	if (context.isCancelOrdered()) { break; }
	// ...
}

Both read the same order. The DB is consulted only once per batch.cancel_check, so calling it every iteration is fine.

Reading a batch of rows at a time#

When you don't want to hold a million rows in one transaction, extend AbstractChunkBatch. It repeats read → process → write in bulk, committing every chunkSize() items.

public class RequestArchiveBatch extends AbstractChunkBatch<Data> {

	@Override public String batchName ()   { return "Archive old requests"; }
	@Override public String cron ()        { return "0 4 * * *"; }
	@Override public int chunkSize ()      { return 500; }

	@Override
	protected Iterator<Data> reader (BatchArgs args, DB db) {

		// read a little at a time, in key order
		return KeyPagingReader.of("id", 0L, chunkSize(), (lastKey, limit) -> db.selectList("""
				SELECT id, amount FROM request
				WHERE status = ? AND id > ?
				ORDER BY id ASC
				LIMIT ?
			""", "approved", lastKey, limit));

	}

	@Override
	protected Data process (Data item) {

		// return null to leave an item out (the default just passes it through)
		return item.getLong("amount") == 0 ? null : item;

	}

	@Override
	protected void write (List<Data> items, DB db) {

		for (Data item : items) {
			db.update("UPDATE request SET status = 'archived' WHERE id = ?", item.getLong("id"));
		}

	}

}

You register it, and see it in the admin screen, exactly like any other batch.

• One call to write() is one transaction. It commits when the method returns
• Cancellation is checked at each chunk boundary. You don't call isCancelOrder() yourself
• execute() is final. The only places you write code are reader(), process() and write()

When it fails partway#

That chunk is rolled back and the whole batch fails. It never quietly moves on.

How far it got is recorded in batch_history.execute_info.

chunk_written is rewritten every batch.progress (5 by default), so the admin screen shows how far along a running batch is.

The reading DB and the writing DB are different#

The db handed to reader() and the db handed to write() are different instances. Use the one you are given.

When a chunk's transaction (Tx) ends the connection goes back to the pool, so if you read through the same DB, your open read dies the moment the first chunk commits.

The key you give KeyPagingReader must be unique, in step with the ORDER BY, and not rewritten while the batch runs. If a full page comes back without the key advancing, it throws — better that than spinning forever in silence.

[!WARNING] When SQL inside write() fails, it throws SqlExecuteException. That chunk is rolled back and the batch fails (see "When it fails partway" above). You do not need to check the result of each statement. Do not catch the failure and carry on. Committing that chunk is then refused with TransactionException (DB_004) and everything in it is rolled back.

Double starts#

How many copies of the same batch may run at once is set by allowConcurrentExecutionCount(). The default is one. If the previous execution has not finished, the next one is skipped.

Even across several servers, one cron firing runs at most as many times as the concurrency limit (since 2.5.8).

Every server sees the same firing time, but only the servers that manage to advance the "last fired" time and the "runs in that firing" count in batch_master run it. Up to 2.5.7 only the number of running copies was checked, so a batch that finished quickly could run again on a server whose clock reached the same firing a little later (the check runs every second, plus any clock drift between servers).

Running one on its own#

You write the batch entry point yourself. The whole startup order is right there in that one file.

public class BlogBatch {

	public static void main (String[] args) {

		DBUtil.load(Conf.conf().config(), BlogBatch.class);   // 1. DB
		BatchTables.install(DBUtil.getMainDB());              // 2. the batch tables

		BatchRegistry.add(PostCleanupBatch::new);             // 3. register
		BatchRegistry.sync(DBUtil.getMainDB());

		BatchResult result = BatchExecutor.start(args);       // 4. run

		DBUtil.stop();
		System.exit(result == BatchResult.completed ? 0 : 1);

	}

}

java -Djimble.env=prod -cp app.jar blog.BlogBatch class=blog.batch.PostCleanupBatch days=30

Anything you pass as key=value lands in args.cliArgs. The environment comes from -Djimble.env (passing env= does not change it, and a value that differs from the running environment stops the batch). Exit with 0 only on completed. isExecuted() means "it ran", and is true for error and canceled too.

To run on cron, stand up a separate resident process with DbScheduler as its entry point.

The admin screen#

Add jimble-batch-manager and you get a web view that lists your batches and lets you run them, cancel them, and read their history.

implementation("io.jimble:jimble-batch-manager:<version>")

install(BatchManagerController::new);

Basic authentication is built in. Turn the screen on in the config and give it a user name and password.

batch_manager {
  enabled  = true                                   # false by default
  username = "admin"
  password = ${?JIMBLE_BATCH_MANAGER_PASSWORD}
}

If the user name or password is empty, the screen is not mounted at all (only an error is logged). It is never published without authentication.