{"id":15104895,"url":"https://github.com/ndegwamoche/php-csv-processor","last_synced_at":"2026-01-18T23:01:41.520Z","repository":{"id":254452706,"uuid":"845085901","full_name":"ndegwamoche/php-csv-processor","owner":"ndegwamoche","description":"This project is a PHP script designed to process a CSV file and insert the data into a PostgreSQL database. The script is executed from the command line and includes various options for creating the database table, running in dry-run mode, and more.","archived":false,"fork":false,"pushed_at":"2024-08-27T17:37:57.000Z","size":264,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T12:42:00.851Z","etag":null,"topics":["cmd","csv","generator","php","postgres","script"],"latest_commit_sha":null,"homepage":"https://github.com/ndegwamoche/php-csv-processor","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ndegwamoche.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-20T14:53:07.000Z","updated_at":"2024-08-27T17:38:01.000Z","dependencies_parsed_at":"2024-08-27T19:12:11.498Z","dependency_job_id":null,"html_url":"https://github.com/ndegwamoche/php-csv-processor","commit_stats":{"total_commits":57,"total_committers":1,"mean_commits":57.0,"dds":0.0,"last_synced_commit":"fdb2bc9ff009bd5a1fbb26c18083297494aa76f6"},"previous_names":["ndegwamoche/php-csv-processor"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ndegwamoche/php-csv-processor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ndegwamoche%2Fphp-csv-processor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ndegwamoche%2Fphp-csv-processor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ndegwamoche%2Fphp-csv-processor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ndegwamoche%2Fphp-csv-processor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ndegwamoche","download_url":"https://codeload.github.com/ndegwamoche/php-csv-processor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ndegwamoche%2Fphp-csv-processor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28553055,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T20:59:07.572Z","status":"ssl_error","status_checked_at":"2026-01-18T20:59:02.799Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cmd","csv","generator","php","postgres","script"],"created_at":"2024-09-25T20:03:39.538Z","updated_at":"2026-01-18T23:01:41.504Z","avatar_url":"https://github.com/ndegwamoche.png","language":"PHP","readme":"# PHP CSV Processor\n\n## Table of Contents\n\n1.  [PHP CSV Processor](#php-csv-processor)\n    1.  [Description](#description)\n    2.  [Tech Stack](#tech-stack)\n        1.  [Dependencies](#dependencies)\n        2.  [Composer Configuration](#composer-configuration)\n    3.  [Design](#design)\n    4.  [Features](#features)\n2.  [How to Run the Project](#how-to-run-the-project)\n    1.  [Prerequisites](#prerequisites)\n    2.  [Setup](#setup)\n        1.  [Clone the Repository](#clone-the-repository)\n        2.  [Install Dependencies](#install-dependencies)\n        3.  [Configure Database Connection](#configure-database-connection)\n    3.  [Usage](#usage)\n    4.  [Example Commands](#example-commands)\n3.  [Functions](#functions)\n    1.  [run()](#run)\n    2.  [countLines()](#countlinesfilename)\n    3.  [validateEmail()](#validateemailemail)\n    4.  [displayProgress()](#displayprogressprocessedlines-totallines-complete--false)\n    5.  [isValidCsvFile()](#isvalidcsvfilefilename)\n    6.  [insertUsers()](#insertusersarray-users-array-errors)\n    7.  [processCSVFile()](#processcsvfile)\n    8.  [createTable()](#createtable)\n    9.  [printError()](#printerrormessage)\n    10. [printInfo()](#printinfo)\n    11. [printHelp()](#printhelp)\n    12. [prompt()](#promptmessage-default--)\n    13. [connectDatabase()](#connectdatabase)\n    14. [parseArguments()](#parseargumentsargv)\n4.  [Error Handling](#error-handling)\n    1.  [Error Messages](#error-messages)\n    2.  [Input Validation](#input-validation)\n    3.  [File Validation](#file-validation)\n    4.  [Database Operations](#database-operations)\n    5.  [Dry Run Mode](#dry-run-mode)\n5.  [Project Wrap-Up](#project-wrap-up)\n6.  [About the Author](#about-the-author)\n\n## Description\n\nThe **PHP CSV Processor** is a command-line script designed for bulk importing user data from CSV files into a PostgreSQL database. This tool streamlines user record management by automating data validation, formatting, and database operations. \n\nIt offers various command-line options for managing database schemas and performing data uploads, making it a versatile solution for database administrators and developers.\n\n## Tech Stack\n\n- **Programming Language**: PHP Version 8.2.x or higher.\n- **Database**: PostgreSQL Version 16.4 or higher.\n- **CSV File**: A properly formatted CSV file containing user data.\n- **Dependency Management**: Composer\n\n### Dependencies\n\nThis project relies on PHP CodeSniffer for maintaining coding standards. PHP CodeSniffer is a tool that helps ensure code adheres to defined coding standards, making it easier to maintain consistency and quality across the codebase.\n\n- **PHP CodeSniffer (`squizlabs/php_codesniffer`)**: A development dependency used to check the code against predefined coding standards and detect issues related to code formatting and style.\n\n  - **Version**: `4.0.x-dev`\n  - **Purpose**: To enforce coding standards and automatically fix some common code style issues.\n\nThe `require-dev` section of the `composer.json` file includes this dependency, which means it is only needed during development and is not required in the production environment.\n\n### Composer Configuration\n\nThe `composer.json` file includes the following configuration for PHP CodeSniffer:\n\n```json\n\"require-dev\": {\n    \"squizlabs/php_codesniffer\": \"4.0.x-dev\"\n}\n```\n\nThis specifies that PHP CodeSniffer is a development dependency, allowing you to run code quality checks and ensure adherence to coding standards during the development process.\n\n## Design\n\nThis project operates as a command-line utility and does not include a graphical user interface. Below is an example of how to interact with the script through the terminal.\n\n![PHP CSV processor image file with a screenshot of the help section](https://raw.githubusercontent.com/ndegwamoche/php-csv-processor/main/php-csv-processor.png)\n\n## Features\n\n- **CSV Parsing** : Import user data from a CSV file into a PostgreSQL database.\n- **Email Validation**: Validates email addresses.\n- **CSV File Validation**: Ensures the CSV file is properly formatted, and contains the correct number and type of columns.\n- **Name Formatting**: Automatically capitalizes the first letter of names and surnames.\n- **Database Table Creation**: Automatically creates or rebuilds the users table in PostgreSQL if needed.\n- **Dry Run Mode**: Allows you to simulate the upload process without modifying the database.\n- **Command-Line Options**: Provides a command-line interface with multiple options for flexibility.\n\n## How to Run the Project\n\n### Prerequisites\n\n- **PHP 8.2.7** or higher\n- **PostgreSQL** database\n- **Composer** for managing dependencies\n\n### Setup\n\n1.  #### **Clone the Repository**\n    ```bash\n    git clone https://github.com/ndegwamoche/php-csv-processor.git\n    cd php-csv-processor\n    ```\n3.  #### **Install Dependencies**\n\n    Ensure you have Composer installed, then run:\n\n    ```bash\n    composer install\n    ```\n4.  #### **Configure Database Connection**\n\n    Database credentials can be specified via command-line options when running the script.\n\n### Usage\n\nHere are the common commands for using the `user_upload.php` script:\n\n- #### **Show Help**\n\n  ```bash\n  php user_upload.php --help\n  ```  \n  Displays available commands and options.\n\n- #### **Create or Rebuild Table**\n\n  ```bash\n  php user_upload.php --create_table\n  ```\n  Creates or rebuilds the `users` table. Optionally provide credentials:\n\n  ```bash\n  php user_upload.php --create_table -u username -p password -h hostname\n  ```  \n- #### **Upload User Data**\n\n  ```bash\n  php user_upload.php --file users.csv\n   ```\n  \n  Uploads data from `users.csv` into the database. Optionally provide credentials:\n\n  ```bash\n  php user_upload.php --file users.csv -u username -p password -h hostname\n  ```\n  \n- #### **Dry Run**\n\n  ```bash\n  php user_upload.php --dry_run\n    ```\n  \n  Simulates the upload process without actual changes. Optionally specify a CSV file:\n\n  ```bash\n  php user_upload.php --dry_run --file users.csv\n  ``` \n   \n### Example Commands\n\n1.  #### **Create the Table**\n\n       ```bash\n    php user_upload.php --create_table -u myuser -p mypass -h localhost\n    ``` \n    \n2.  #### **Upload Data**\n\n     ```bash\n    php user_upload.php --file users.csv -u myuser -p mypass -h localhost\n     ``` \n    \n3.  #### **Dry Run Upload**\n\n     ```bash\n    php user_upload.php --dry_run --file users.csv\n     ``` \n    \n## Functions\n\n### `run()`\n\n**Main function to execute the script based on the parsed arguments.**\n\n- **Description**: This method handles the overall execution of the script. It checks the provided command-line arguments and performs actions such as creating the database table, processing the CSV file, or running in dry mode. It ensures that the necessary actions are taken based on user input.\n- **Usage**:\n\n  ```php\n\n  // The `run` method is automatically called based on the command-line arguments.\n  ```    \n### `countLines($filename)`\n\n**Counts the number of lines in a specified file.**\n\n- **Parameters**:\n  - `string $filename`: The name of the file to count lines in.\n- **Returns**: `int` - The total number of lines in the file.\n- **Usage**:\n\n  ```php\n  $lineCount = countLines('users.csv');\n  ```\n  \n### `validateEmail($email)`\n\n**Validates the format of an email address.**\n\n- **Parameters**:\n  - `string $email`: Email address to validate.\n- **Returns**: `bool` - `true` if the email is valid, `false` otherwise.\n- **Usage**:\n\n  ```php\n  $isValid = validateEmail('test@example.com');\n  ```\n    \n### `displayProgress($processedLines, $totalLines, $complete = false)`\n\n**Displays a progress bar in the command line to indicate processing status.**\n\n- **Parameters**:\n  - `int $processedLines`: The number of lines processed so far.\n  - `int $totalLines`: The total number of lines to process.\n  - `bool $complete`: Optional parameter to indicate if the processing is complete.\n- **Usage**:\n\n  ```php\n  displayProgress(50, 100);\n  displayProgress(100, 100, true);\n  ```\n    \n### `isValidCsvFile($filename)`\n\n**Validates if the given file is a proper CSV file.**\n\n- **Description**: This function checks several criteria to confirm the file's validity, including existence, readability, file extension, MIME type, and consistent column count across rows.\n- **Parameters**:\n  - `string $filename`: The path to the file to be validated.\n- **Returns**: `bool` - `true` if the file is a valid CSV, `false` otherwise.\n- **Usage**:\n\n  ```php\n  $isValid = isValidCsvFile('path/to/file.csv');\n  ```\n  \n### `insertUsers(array $users, array \u0026$errors)`\n\n**Inserts multiple user records into the database.**\n\n- **Parameters**:\n  - `array $users`: An array of associative arrays, each containing 'name', 'surname', and 'email' keys for user records.\n  - `array \u0026$errors`: An array that will be populated with errors encountered during insertion.\n- **Usage**:\n\n  ```php\n  insertUsers($userRecords, $errors);\n  ```\n\n### `processCSVFile()`\n\n**Processes a CSV file and inserts valid user records into the database.**\n\n- **Description**: This method handles reading and validating the CSV file, processing each row, and inserting valid user records into the database. It also displays appropriate messages for invalid data.\n- **Usage**:\n\n ```php\n  processCSVFile();\n ```\n### `createTable()`\n\n**Creates the PostgreSQL users table.**\n\n- **Description**: This method creates a 'users' table in the PostgreSQL database with columns for ID, name, surname, and email. The email column must be unique.\n- **Usage**:\n\n  ```php\n  createTable();\n     ```\n  \n### `printError($message)`\n\n**Prints error messages in red.**\n\n- **Parameters**:\n  - `string $message`: Message to display.\n- **Usage**:\n\n  ```php\n  printError(\"An error occurred.\");\n     ```\n### `printInfo($message)`\n\n**Prints informational messages in green.**\n\n- **Parameters**:\n  - `string $message`: Message to display.\n- **Usage**:\n\n  ```php\n  printInfo(\"Process completed successfully.\");\n     ```\n  \n### `printHelp()`\n\n**Displays help information about how to use the script.**\n\n- **Description**: This method provides information on how to use the script, including available commands and options.\n- **Usage**:\n\n ```php\n  printHelp();\n ```\n\n### `prompt($message, $default = '')`\n\n**Prompts the user for input and returns the entered value.**\n\n- **Parameters**:\n  - `string $message`: The message to display to the user.\n  - `string $default`: The default value to return if no input is provided.\n- **Returns**: `string` - The user's input or the default value if no input is provided.\n- **Usage**:\n\n  ```php\n  $input = prompt(\"Enter PostgreSQL host (default: localhost):\", \"localhost\");\n  ```\n\n### `connectDatabase()`\n\n**Establishes a connection to the PostgreSQL database using PDO.**\n\n- **Description**: This method creates a PDO connection to the PostgreSQL database, enabling interaction with the database for operations such as table creation and data insertion.\n- **Usage**:\n\n  ```php\n  connectDatabase();\n  ```\n\n### `parseArguments($argv)`\n\n**Parses command-line arguments.**\n\n- **Parameters**:\n  - `array $argv`: Command-line arguments.\n- **Returns**: `array` - Parsed arguments.\n- **Usage**:\n\n   ```php\n  $args = parseArguments($argv);\n    ```\n\n## Error Handling\n\nThe PHP CSV Processor script includes several mechanisms for error handling to ensure smooth execution and provide clear feedback when issues arise. Here’s an overview of how errors are managed within the script:\n\n### Error Messages\n\n- **Error Printing**: Error messages are printed in red to distinguish them from other types of output. This helps in quickly identifying problems when running the script from the command line.\n\n  ```php\n  private function printError($message) {\n    // Print error messages in red\n    echo \"\\033[31m$message\\033[0m\\n\";\n  }\n     ```\n\n- **Informational Messages**: Informational messages are printed in green to provide feedback on the progress and status of the script.\n\n  ```php\n  private function printInfo($message) {\n    // Print informational messages in green\n    echo \"\\033[32m$message\\033[0m\\n\";\n  }\n   ```\n\n### Input Validation\n\n- **Command-Line Arguments**: The script checks if the necessary command-line arguments are provided. If not, it displays an error message and prints the help information. This ensures that users know how to properly run the script.\n\n ```php\n  if (empty($this-\u003eargs)) {\n    $this-\u003eprintError(\"Please enter a command to start.\\nPlease use the options below to run the script.\");\n    $this-\u003eprintHelp();\n    exit(1);\n  }\n ```\n\n### File Validation\n\n- **CSV File Validation**: The script validates the CSV file to ensure it meets several criteria:\n\n  - The file exists and is readable.\n  - The file has a `.csv` extension.\n  - The MIME type of the file matches that of a CSV file.\n  - The file is parsed to verify consistent column count across rows.\n\n  If the file does not meet these criteria, the script will output an error message and halt further processing.\n\n ```php\n  private function isValidCsvFile($filename) {\n    // Validate CSV file\n    if (!file_exists($filename)) {\n        $this-\u003eprintError(\"File does not exist.\");\n        return false;\n    }\n    if (!is_readable($filename)) {\n        $this-\u003eprintError(\"File is not readable.\");\n        return false;\n    }\n    if (pathinfo($filename, PATHINFO_EXTENSION) !== 'csv') {\n        $this-\u003eprintError(\"File is not a CSV.\");\n        return false;\n    }\n    // Additional validation logic\n    return true;\n}\n```\n\n### Database Operations\n\n- **Database Connection**: The script establishes a connection to the PostgreSQL database. If connection fails, an error message is printed.\n\n  ```php\n  private function connectDatabase() {\n    try {\n        // Database connection logic\n    } catch (PDOException $e) {\n        $this-\u003eprintError(\"Database connection failed: \" . $e-\u003egetMessage());\n        exit(1);\n    }\n  }\n  ```\n\n- **Record Insertion**: Errors during the insertion of user records into the database are handled individually. The script logs errors for specific records, allowing the remaining records to be processed.\n\n ```php\n  private function insertUsers(array $users, array \u0026$errors) {\n    foreach ($users as $user) {\n        try {\n            // Insert user record\n        } catch (Exception $e) {\n            // Handle the error for this specific user and add it to the errors array\n            $errors[] = \"Failed to insert user: \" . $e-\u003egetMessage();\n        }\n    }\n  }\n```\n\n### Dry Run Mode\n\n- **Dry Run Mode**: When running in dry-run mode, the script performs a simulated execution without making any changes to the database. This is useful for verifying the results before actual execution.\n\n  ```php\n  if (isset($this-\u003eargs['dry_run'])) {\n    $this-\u003eprintInfo(\"Dry run mode enabled. No changes will be made to the database.\");\n    $this-\u003eprocessCSVFile();\n  }\n  ```\n## Project Wrap-Up\n\nIn conclusion, this project successfully demonstrates the integration of various technologies and best practices to create a robust and efficient solution for processing and managing CSV data in a PostgreSQL database. By leveraging PHP's capabilities and adhering to industry standards, the project not only ensures data integrity and security but also provides a flexible and scalable framework for future enhancements. The implementation of detailed error handling, user authentication, and seamless database operations reflects a deep commitment to quality and performance. This project serves as a solid foundation for further development and showcases the importance of well-structured code and thoughtful design in achieving reliable software solutions.\n\n## About the Author\n\n**Martin Ndegwa Moche**\n\nI am a WordPress PHP Developer with a passion for building robust and efficient web applications. If you have any questions or would like to connect, feel free to reach out!\n\n- **LinkedIn**: [Martin Ndegwa Moche](https://www.linkedin.com/in/ndegwamoche/)\n- **Email**: ndegwamoche@gmail.com\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fndegwamoche%2Fphp-csv-processor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fndegwamoche%2Fphp-csv-processor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fndegwamoche%2Fphp-csv-processor/lists"}