Breaking CLI Commands into Chunked Web APIs to Avoid Timeouts

Long-running CLI commands that work perfectly in terminal contexts often fail spectacularly in web environments. The culprit? Execution time limits and connection timeouts.

Consider a typical bulk operation command:

// CLI command - works but unusable in web context
public function handle()
{
    DB::beginTransaction();
    
    $items = Item::whereIn('id', $this->option('items'))->get();
    
    foreach ($items as $item) {
        $item->update(['status' => 'maintenance']);
        $this->processRelatedRecords($item);
    }
    
    DB::commit();
    $this->info('Done!');
}

This works great in Artisan but dies immediately when exposed via web UI – browser timeouts, server limits, no progress feedback.

The Progressive Web API Pattern

Break the monolithic operation into two separate endpoints:

Step 1: Initialization Endpoint

Validates input and creates tracking record:

public function executeStart(Request $request): JsonResponse
{
    $validated = $request->validate([
        'item_ids' => 'required|array',
        'item_ids.*' => 'exists:items,id',
    ]);
    
    $items = Item::whereIn('id', $validated['item_ids'])->get();
    $snapshot = $this->buildInitialSnapshot($items);
    
    $revision = Revision::create([
        'user_id' => auth()->id(),
        'key' => 'batch_maintenance',
        'old_value' => $snapshot,
        'new_value' => $snapshot,
    ]);
    
    return response()->json([
        'status' => 'success',
        'data' => [
            'revision_id' => $revision->id,
            'items' => $items->map(fn($i) => [
                'id' => $i->id,
                'title' => $i->title,
            ]),
        ],
    ]);
}

Step 2: Per-Item Execution Endpoint

Processes ONE item per request:

public function executeItem(Request $request): JsonResponse
{
    $validated = $request->validate([
        'revision_id' => 'required|exists:revisions,id',
        'item_id' => 'required|exists:items,id',
    ]);
    
    $revision = Revision::findOrFail($validated['revision_id']);
    $item = Item::findOrFail($validated['item_id']);
    
    try {
        $item->update(['status' => 'maintenance']);
        $this->processRelatedRecords($item);
        
        $this->updateRevisionResult($revision, $item->id, 'success');
        
        return response()->json([
            'status' => 'success',
            'data' => ['item_id' => $item->id, 'result' => 'success'],
        ]);
    } catch (\Exception $e) {
        $this->updateRevisionResult($revision, $item->id, 'failed', $e->getMessage());
        
        return response()->json([
            'status' => 'error',
            'data' => ['item_id' => $item->id, 'result' => 'failed', 'error' => $e->getMessage()],
        ], 422);
    }
}

Why This Works

Each request is fast. No timeout issues – every API call completes in milliseconds.

Real-time progress. Frontend shows “Processing 5 of 20…” as each request completes.

Partial failures don’t lose everything. If item #15 fails, items 1-14 are already committed.

User stays in control. Can pause/resume, see exactly what succeeded vs failed.

The Trade-Off

You lose transactional atomicity – it’s no longer “all or nothing”. But in practice, this is acceptable for most bulk operations where seeing incremental progress and recovering from partial failures matters more than database transaction boundaries.

For operations that truly must be atomic, keep them CLI-only. For everything else, this pattern transforms unusable monolithic commands into production-ready progressive workflows.