Import Large CSV File into Database Using Laravel

Importing a large CSV file into a database in Laravel can be a complex task due to memory and performance considerations. Here's a step-by-step guide on how to achieve this efficiently:

1. Install Laravel Excel:

Run the following command to install the Laravel Excel package.

composer require maatwebsite/excel

2. Create a Migration:

Create a migration to define the structure of the table where you want to store the CSV data.

php artisan make:migration create_csv_data_table

Edit the migration file to define the columns you want to import.

// database/migrations/xxxx_xx_xx_create_csv_data_table.php

use Illuminate\Database\Migrations\Migration;

use Illuminate\Database\Schema\Blueprint;

use Illuminate\Support\Facades\Schema;

class CreateCsvDataTable extends Migration

{

    public function up()

    {

        Schema::create('csv_data', function (Blueprint $table) {

            $table->id();

            $table->string('column1');

            $table->string('column2');

            // ... Define other columns

            $table->timestamps();

        });

    }

    public function down()

    {

        Schema::dropIfExists('csv_data');

    }

}

Run the migration.

php artisan migrate

3. Create Import Class:

Generate an import class to handle the CSV import.

php artisan make:import CsvImport

Edit the `CsvImport` class to map the CSV columns to your model's attributes.

// app/Imports/CsvImport.php

namespace App\Imports;

use Maatwebsite\Excel\Concerns\ToModel;

use Maatwebsite\Excel\Concerns\WithChunkReading;

use App\Models\CsvData;

class CsvImport implements ToModel, WithChunkReading

{

    public function model(array $row)

    {

        return new CsvData([

            'column1' => $row[0],

            'column2' => $row[1],

            // ... Map other columns

        ]);

    }

    public function chunkSize(): int

    {

        return 1000; // Adjust chunk size as needed

    }

}

4. Create Controller:

Create a controller to handle the import process.

php artisan make:controller CsvImportController

Edit the `CsvImportController` to include the import logic.

// app/Http/Controllers/CsvImportController.php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

use Maatwebsite\Excel\Facades\Excel;

use App\Imports\CsvImport;

class CsvImportController extends Controller

{

    public function import(Request $request)

    {

        $file = $request->file('csv_file');

        Excel::import(new CsvImport, $file);

        return redirect()->back()->with('success', 'CSV file imported successfully.');

    }

}

5. Create a Route:

Define a route for the import action.

// routes/web.php

use App\Http\Controllers\CsvImportController;

Route::post('/import', [CsvImportController::class, 'import'])->name('import');

6. Create View:

Create a view to upload the CSV file.

<!-- resources/views/import.blade.php -->

<form action="{{ route('import') }}" method="POST" enctype="multipart/form-data">

    @csrf

    <input type="file" name="csv_file">

    <button type="submit">Import</button>

</form>

7. Run Development Server:

Run the Laravel development server.

php artisan serve

Now, when you access the view you've created (e.g., `http://localhost:8000/import`), you can upload the CSV file and it will be imported into the database using the Laravel Excel package.

Remember to replace placeholders like `App\` with your actual namespace and update the model and column names as per your application's requirements.