{"id":24982399,"url":"https://github.com/ahnineamine/log-analysis-api","last_synced_at":"2025-03-29T09:46:06.715Z","repository":{"id":275336628,"uuid":"925784686","full_name":"ahnineamine/log-analysis-api","owner":"ahnineamine","description":"OCaml-based REST API designed to process, analyze, and transform log data","archived":false,"fork":false,"pushed_at":"2025-02-01T18:53:45.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-01T19:32:11.635Z","etag":null,"topics":["dune","functional-programming","ocaml"],"latest_commit_sha":null,"homepage":"","language":"OCaml","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahnineamine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-01T18:34:22.000Z","updated_at":"2025-02-01T18:57:22.000Z","dependencies_parsed_at":"2025-02-01T19:42:15.233Z","dependency_job_id":null,"html_url":"https://github.com/ahnineamine/log-analysis-api","commit_stats":null,"previous_names":["ahnineamine/log-analysis-api"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahnineamine%2Flog-analysis-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahnineamine%2Flog-analysis-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahnineamine%2Flog-analysis-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahnineamine%2Flog-analysis-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahnineamine","download_url":"https://codeload.github.com/ahnineamine/log-analysis-api/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246168087,"owners_count":20734389,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dune","functional-programming","ocaml"],"created_at":"2025-02-04T08:43:30.154Z","updated_at":"2025-03-29T09:46:06.692Z","avatar_url":"https://github.com/ahnineamine.png","language":"OCaml","readme":"# Log Analysis API\n\nLog Analysis API is an OCaml-based REST API designed to process, analyze, and transform log data. It demonstrates the power of OCaml's functional programming paradigm, type safety, and performance for real-world data processing tasks.\n\n## Features\n\n- **Filtering**: Filter logs by log level (e.g., INFO, WARN, ERROR, DEBUG) or by substrings in the log message.\n- **Aggregation**: Compute aggregates such as count, average, sum, max, and min for numeric log fields.\n- **Transformation**: Convert logs into JSON or CSV formats.\n- **Schema Endpoint**: Expose a JSON schema that describes the expected format for log entries.\n- **Health Checks**: Basic endpoints to verify service liveness and readiness.\n\n## Prerequisites\n\n- **OCaml** (version 4.08 or later recommended)\n- **Dune** build system\n- **Opam** package manager\n\n### Required Libraries\n\n- `dream`\n- `lwt`\n- `yojson`\n- `lwt_ppx` (for PPX syntax, e.g., `let%lwt`)\n\nYou can install these using opam:\n\n```sh\nopam install dream lwt yojson lwt_ppx\n```\n\n## Installation and Running\n### Clone the repository:  \n```sh\ngit clone https://github.com/ahnineamine/log-analysis-api.git\ncd log-analysis-api\n```\n### Build the project:  \n```sh\ndune build\n```\n\n### Run the API server:\n```sh\ndune exec ./your_executable_name\n```\n\nThe server will start (by default on port 8080) and you can access the endpoints at http://localhost:8080/.\n\n## API Endpoints\n### POST /process\nProcesses a command on the logs loaded from the `data/` folder. The request payload should contain a command. Since logs are read from the source and kept in memory, you only need to pass the command.\n\nExample Payloads:\n\n#### Filter by Log Level (\"ERROR\"):  \n```json\n{\n    \"type\": \"Filter\",\n    \"criteria\": {\n        \"level\": \"ERROR\"\n    }\n}\n```\n\n#### Filter by Substring in Message (\"failed\"):  \n```json\n{\n    \"type\": \"Filter\",\n    \"criteria\": {\n        \"message_contains\": \"failed\"\n    }\n}\n```\n\n#### Aggregate (Count by \"user_id\"):\n```json\n{\n    \"type\": \"Aggregate\",\n    \"criteria\": {\n        \"operation\": \"CountBy\",\n        \"field\": \"user_id\"\n    }\n}\n```\n\n#### Transform to CSV:\n```json\n{\n  \"type\": \"Transform\",\n  \"format\": \"CSV\"\n}\n```\n\n### GET /schema\nReturns a JSON schema describing the expected log format, including standard fields like timestamp, level, message, and metadata.\n\n### GET /health\nA basic health check endpoint that returns the status of the API.\n\n## Log Files\nPlace your log files (pretty-printed JSON arrays) in the data/ folder. Sample log files are provided in this folder for testing. Each file should contain a valid JSON array of log objects.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahnineamine%2Flog-analysis-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahnineamine%2Flog-analysis-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahnineamine%2Flog-analysis-api/lists"}