{"id":13565327,"url":"https://github.com/dvoraka/av-service","last_synced_at":"2026-01-16T10:27:55.058Z","repository":{"id":77294226,"uuid":"51168229","full_name":"dvoraka/av-service","owner":"dvoraka","description":"Network file service with replication and anti-virus checking","archived":false,"fork":false,"pushed_at":"2020-02-14T14:08:37.000Z","size":4016,"stargazers_count":10,"open_issues_count":37,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-04T18:46:43.564Z","etag":null,"topics":["amqp","anti-virus","database","file-services","java","jms","kafka","replication","rest","solr","spring","spring-boot","storage"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dvoraka.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-02-05T19:14:04.000Z","updated_at":"2021-06-11T17:03:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"9b5f148a-f114-42e8-9c6f-49e2a832db9b","html_url":"https://github.com/dvoraka/av-service","commit_stats":null,"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvoraka%2Fav-service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvoraka%2Fav-service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvoraka%2Fav-service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvoraka%2Fav-service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dvoraka","download_url":"https://codeload.github.com/dvoraka/av-service/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247090065,"owners_count":20881912,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amqp","anti-virus","database","file-services","java","jms","kafka","replication","rest","solr","spring","spring-boot","storage"],"created_at":"2024-08-01T13:01:44.690Z","updated_at":"2026-01-16T10:27:55.038Z","avatar_url":"https://github.com/dvoraka.png","language":"Java","funding_links":[],"categories":["Java"],"sub_categories":[],"readme":"# Network file service with anti-virus checking\n[![Build Status](https://travis-ci.org/dvoraka/av-service.svg?branch=master)](https://travis-ci.org/dvoraka/av-service)\n[![codecov.io](https://codecov.io/github/dvoraka/av-service/coverage.svg)](https://codecov.io/github/dvoraka/av-service/branch/master)\n[![Latest release](https://img.shields.io/badge/release-0.10-brightgreen.svg)](https://github.com/dvoraka/av-service/releases/tag/v0.10)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/2e3856220448493482f6de90a0d84ee0)](https://www.codacy.com/app/dvoraka/av-service?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=dvoraka/av-service\u0026amp;utm_campaign=Badge_Grade)\n[![Open Hub](https://www.openhub.net/p/av-service/widgets/project_thin_badge?format=gif)](https://www.openhub.net/p/av-service/)\n[![IntelliJ IDEA](/imgs/idea_logo.png)](https://www.jetbrains.com)\n\nA file service prototype with a replication and anti-virus checking. It is mostly a study project\nbut it is possible to use it after some \"production\" tuning.\n\nA replacement for [amqpav](https://github.com/dvoraka/amqpav) and first requirements were from\nthe old service.\nActual network file service with anti-virus checking supports JMS, AMQP, Kafka and REST for\ntransport.\nAV-checker project was integrated as the checker submodule, then completely rewritten and\nthe old code was removed.\n\nFrom version 0.6 it is mostly a file service with an AV checking.\nCurrently it is still a bit experimental feature.\nSo if you need the AV checking only you can use the maintained 0.5 version.\n\n### Contribution\nPossibilities: Quality assurance (QA), Front-end for REST services (statistics and file storage),\nLinux admin for better deployment and distribution, Wiki mage, Server for testing, or we\ncan find something else for you\n\n---\n\n * [Release notes](#release-notes)\n * [Features](#features)\n * [Planned features](#planned-features)\n * [Overview](#overview)\n * [Used components](#used-components)\n * [How to send a message](#how-to-send-a-message-for-anti-virus-check)\n * [Installation](#installation)\n * [Run services](#run-service)\n * [AMQP checker](#amqp-checker)\n * [JMS checker](#jms-checker)\n * [Legacy checker](#old-checker)\n * [Load tests](#load-tests)\n\n---\n\n### Release notes\nRelease [NOTES](RELEASE_NOTES.md)\n\n### Features\n * AMQP 0.9.1 support\n * JMS support\n * Kafka support\n * REST support\n * Message tracking DB service (PostgreSQL or Solr)\n * Statistics module\n * File service with network replication\n * Highly scalable architecture\n * Robust design\n \n### Planned features\n * Separate REST app for better scaling\n * REST load balancer\n * AMQP 1.0 support\n\n### Overview\n##### Remote REST configuration with file replication\n![Remote REST component diagram](/docs/remoterestcomponent.png)\n\n##### Broker connection configuration with file replication\n![Broker connection component diagram](/docs/brokercomponent.png)\n\n##### Local REST connection configuration\n![Local REST component diagram](/docs/localrestcomponent.png)\n\n### Used components\n * **ClamAV** - open source anti-virus engine\n * **RabbitMQ** - open source message broker used for AMQP\n * **ActiveMQ** - open source message broker used for JMS\n * **Kafka** - open source message broker used for Kafka\n * **PostgreSQL** - open source DB\n * **Solr** - open source enterprise search platform\n\n**Frameworks and libraries**\n * **Spring**\n * **Spring Boot**\n * **Spring REST** - REST\n * **Spring Security** - REST security\n * **Spring AMQP** - AMQP\n * **Spring JMS** - JMS\n * **Spring Kafka** - Kafka\n * **Spring Data** - message logging and file saving\n * **Ehcache 3** - caching\n * **Spock** - testing\n * **Log4j 2** - logging API\n * **Logback** - logging\n\n**Code quality**\n * **Checkstyle**\n * **FindBugs**\n * **PMD**\n * **JaCoCo**\n * **SonarQube**\n\n**Build**\n * **Gradle** - build\n * **Docker** - environment preparation\n\n**Development**\n * **Elastic Stack** (Elasticsearch, Logstash, Kibana) - logs analyzing\n\n### Spring profiles\nApplication is configurable and it's possible to run many configurations. Here is a list\nwith all profiles and descriptions.\n\nprofile | description | from version | to version\n---|---|---|---\namqp | enable AMQP infrastructure | |\namqp2jms | bridging AMQP to JMS | | 0.6\namqp-checker | AMQP checker | | 0.6\namqp-file-server | AMQP file server | 0.6 | 0.6\namqp-rest | AMQP infrastructure for REST | 0.5 | 0.6\namqp-server | AMQP server | | 0.6\nbridge | enable bridge | 0.7 |\ncheck | enable AV checking | 0.7 |\nchecker | enable checker | 0.7 |\nclient | enable client infrastructure | 0.7 |\ncore | core functionality\ndb | use SQL DB for message logging\ndb-mem | use in-memory SQL DB for message logging | 0.8\ndb-solr | use Solr for message logging\nfile-client | enable file client | 0.7 |\nitest | profile for integration testing | 0.7 |\njms | enable JMS infrastructure\njms2amqp | bridging JMS to AMQP | | 0.6\njms-checker | JMS checker | | 0.6\njms-rest | JMS infrastructure for REST | 0.5 | 0.6\njms-server | JMS server | | 0.6\nldap | enable LDAP infrastructure | 0.7 |\nno-db | disable message logging\nperformance | profile for performance testing | 0.8\nreplication | enable replication infrastructure | 0.7 |\nreplication-test | enable replication testing infrastructure | 0.7 |\nrest | enable REST\nrest-amqp | REST to AMQP | 0.5\nrest-jms | REST to JMS | 0.5\nrest-local | REST with direct connection | 0.5\nserver | enable server infrastructure | 0.7 |\nstats | enable statistics | 0.5\nstats-solr | statistics in Solr | 0.5\nstorage | enable file service | 0.6\nstorage-check | profile for storage testing | 0.8\nto-amqp | bridging to AMQP | 0.7 |\nto-jms | bridging to JMS | 0.7 |\n\nHere will be the best combinations for various usages soon.\n\nYou can find all profiles for a concrete code base with the script **findAllSpringProfiles.sh**\nin the ```tools``` directory.\n\n### How to send a message for anti-virus check\n\nAvCheckExample class:\n```java\npackage dvoraka.avservice.client.example;\n\nimport dvoraka.avservice.client.AvMessageFuture;\nimport dvoraka.avservice.client.configuration.ClientConfig;\nimport dvoraka.avservice.client.service.AvServiceClient;\nimport dvoraka.avservice.common.util.Utils;\nimport dvoraka.avservice.common.data.AvMessage;\nimport org.springframework.context.annotation.AnnotationConfigApplicationContext;\n\n/**\n * Anti-virus checking example.\n */\npublic final class AvCheckExample {\n\n    public static void main(String[] args) throws InterruptedException {\n        // initialize client context\n        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();\n        context.getEnvironment().setActiveProfiles(\"client\", \"amqp\", \"file-client\", \"no-db\");\n        context.register(ClientConfig.class);\n        context.refresh();\n\n        // get client\n        AvServiceClient avServiceClient = context.getBean(AvServiceClient.class);\n\n        // generate message\n        AvMessage avMessage = Utils.genMessage();\n\n        // send it and get response\n        AvMessage response;\n        try {\n            AvMessageFuture futureResponse = avServiceClient.checkMessage(avMessage);\n            response = futureResponse.get();\n        } finally {\n            context.close();\n        }\n\n        // raw output\n        System.out.println(\"Response: \" + response);\n        // virus info\n        System.out.println(\"Virus info: \" + (response != null ? response.getVirusInfo() : \"\"));\n    }\n}\n\n```\n\n### Installation\nYou can use Docker to prepare necessary services for a development.\nThe Docker configurations are a good documentation for installing services without Docker.\nEverything is inside the ```docker/``` directory.\n\n#### Docker\nDocker is recommended approach.\n\n**Services**\n* RabbitMQ\n* ActiveMQ\n* ClamAV daemon\n* PostgreSQL\n* Solr\n* Kibana\n* Elasticsearch\n* Logstash\n* 2 app replication nodes\n* SonarQube (optional)\n\nChange your working directory to ```docker/```:\n```\n$ cd docker/\n```\nAnd run ```docker-compose up``` command:\n```\n$ docker-compose up\n```\nIt prepares services running on **localhost**.\n\n**Configuration:**\n\nFor setting environment it is necessary to run a command from the **root** directory:\n\nchange to project root\n```\n$ cd ..\n```\nand\n```\n$ ./gradlew configureEnvironment\n```\nAnd everything should be prepared for application running.\n\n### Run service\nYou can run all services easily with Gradle.\n\n#### AMQP\nAnti-virus checking server:\n```\n$ ./gradlew runAmqpServer\n```\nFile server with anti-virus checking:\n```\n$ ./gradlew runAmqpFileServer\n```\n#### JMS\nAnti-virus checking server:\n```\n$ ./gradlew runJmsServer\n```\nFile server with anti-virus checking:\n```\n$ ./gradlew runJmsFileServer\n```\n#### REST\nIt is a Spring Boot application. You can run it with Gradle or use an executable jar.\n\n**Gradle**\n```\n$ ./gradlew rest:bootRun\n```\n**Spring Boot**\n\nYou need to build an executable jar:\n```\n$ ./gradlew assemble\n```\nAnd then the the jar is in `rest/build/libs/` directory called **avservice-rest-XXX.jar**.\nYou can run it with:\n```\n$ java -jar avservice-rest-0.9.jar \n\n  .   ____          _            __ _ _\n /\\\\ / ___'_ __ _ _(_)_ __  __ _ \\ \\ \\ \\\n( ( )\\___ | '_ | '_| | '_ \\/ _` | \\ \\ \\ \\\n \\\\/  ___)| |_)| | | | | || (_| |  ) ) ) )\n  '  |____| .__|_| |_|_| |_\\__, | / / / /\n =========|_|==============|___/=/_/_/_/\n :: Spring Boot ::            (v1.5.4.RELEASE)\n```\n\n### AMQP checker\nUtility for testing AMQP infrastructure. It sends a message and wait for a response.\n\n#### Run\n```\n$ ./gradlew -q runAmqpCheck\n```\n\n### JMS checker\nUtility for testing JMS infrastructure. It sends a message and wait for a response.\n\n#### Run\n```\n$ ./gradlew -q runJmsCheck\n```\n\n### Check output logs\n\n```\n12:20:00.388 [main] INFO  d.avservice.client.checker.CheckApp - Checking... \n12:20:03.807 [main] INFO  d.avservice.client.checker.CheckApp - failed!\n12:20:03.808 [main] INFO  d.avservice.client.checker.CheckApp - Check completed.\n```\nor\n```\n12:20:00.388 [main] INFO  d.avservice.client.checker.CheckApp - Checking... \n12:20:03.807 [main] INFO  d.avservice.client.checker.CheckApp - OK\n12:20:03.808 [main] INFO  d.avservice.client.checker.CheckApp - Check completed.\n```\n\n---\n\n### Old checker\nYou can find old checker under **legacy-checker** release.\n\n### AMQP checker\nUtility for testing AMQP infrastructure.\n\n#### Run\n```\n$ ./gradlew -q amqpCheck\n```\nAnd output should be:\n```\n...\nTest OK\n```\n#### Create jar with dependencies\n```\n$ ./gradlew :checker:shadowJar\n```\nAnd the full jar will be in `checker/build/libs/` directory.\n\n#### Run jar\nBuilt jar is executable.\n\n##### Properties:\n* host - message broker host\n* infected - infection flag (true/false)\n* appid - application ID string\n```\n$ java -Dhost=localhost -jar checker-0.1-SNAPSHOT-all.jar\n```\nAnd output should be:\n```\n...\nTest OK\n```\n\n## Load tests\nVarious performance testing:\n[TESTS](performance.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvoraka%2Fav-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdvoraka%2Fav-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvoraka%2Fav-service/lists"}