Queue Large Reports, Don’t Block HTTP Requests

📖 2 minutes read

The Problem

Your users request a large CSV export of 50,000 orders. You build the file synchronously in a controller:

public function export()
{
    $file = fopen('php://temp', 'w+');
    
    Order::chunk(1000, function ($orders) use ($file) {
        foreach ($orders as $order) {
            fputcsv($file, [$order->id, $order->total, $order->status]);
        }
    });
    
    rewind($file);
    return response()->stream(function() use ($file) {
        fpassthru($file);
    }, 200, [
        'Content-Type' => 'text/csv',
        'Content-Disposition' => 'attachment; filename="orders.csv"',
    ]);
}

The request times out after 30 seconds. The user sees a 504 Gateway Timeout error.

The Fix

Dispatch a queued job that emails the file when it’s ready:

// Controller
public function export(Request $request)
{
    GenerateOrdersReport::dispatch($request->user());
    
    return back()->with('success', 'Report is being generated. We\'ll email you when it\'s ready.');
}

// app/Jobs/GenerateOrdersReport.php
class GenerateOrdersReport implements ShouldQueue
{
    public $timeout = 600; // 10 minutes
    public $tries = 1;
    
    public function __construct(
        private User $user
    ) {}
    
    public function handle()
    {
        ini_set('memory_limit', '512M');
        
        $filename = 'orders-' . now()->format('Y-m-d-His') . '.csv';
        $path = storage_path('app/exports/' . $filename);
        
        $file = fopen($path, 'w');
        fputcsv($file, ['Order ID', 'Total', 'Status']); // header
        
        Order::chunk(1000, function ($orders) use ($file) {
            foreach ($orders as $order) {
                fputcsv($file, [$order->id, $order->total, $order->status]);
            }
        });
        
        fclose($file);
        
        $this->user->notify(new ReportReady($filename));
    }
}

The notification email includes a download link:

// app/Notifications/ReportReady.php
class ReportReady extends Notification
{
    public function __construct(
        private string $filename
    ) {}
    
    public function via($notifiable)
    {
        return ['mail'];
    }
    
    public function toMail($notifiable)
    {
        $url = route('reports.download', $this->filename);
        
        return (new MailMessage)
            ->subject('Your report is ready')
            ->line('Your export has been generated.')
            ->action('Download Report', $url)
            ->line('This link expires in 24 hours.');
    }
}

Why It Works

  • No timeout — The job can run for minutes without hitting HTTP limits
  • User feedback — Instant response (“We’re working on it”) instead of a hanging request
  • Memory control — Explicit ini_set('memory_limit') prevents runaway processes
  • Retry safety$tries = 1 prevents duplicate reports if the job fails midway

Best Practices

  1. Set explicit timeouts — Don’t let jobs run forever
  2. Clean up old files — Schedule a daily job to delete exports older than 7 days
  3. Rate limit export requests — Prevent users from queuing 10 exports at once
  4. Track job status — Store $job->getJobId() in the database so you can show “In Progress” UI

For any operation that takes more than 5 seconds, move it to a queue. Users appreciate instant feedback more than waiting for a loading spinner.

Daryle De Silva

VP of Technology

11+ years building and scaling web applications. Writing about what I learn in the trenches.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *