{"id":14956804,"url":"https://github.com/aidanwhiteley/books","last_synced_at":"2025-04-05T12:04:20.876Z","repository":{"id":27911887,"uuid":"115548044","full_name":"aidanwhiteley/books","owner":"aidanwhiteley","description":"A demo project for Spring Boot / Data / security, social / oauth2 logons, JWT, Mongo, SpringBootAdmin, Docker, docker-compose, Github Actions and stateless apps","archived":false,"fork":false,"pushed_at":"2024-10-28T21:27:49.000Z","size":15298,"stargazers_count":100,"open_issues_count":2,"forks_count":26,"subscribers_count":6,"default_branch":"develop","last_synced_at":"2024-10-29T14:21:11.031Z","etag":null,"topics":["bookclub","books","docker","docker-compose","github-actions","jwt","microservice","mongo","oauth","spring-boot","spring-security","springbootadmin","swagger"],"latest_commit_sha":null,"homepage":"https://cloudybookclub.com/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aidanwhiteley.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-27T18:39:06.000Z","updated_at":"2024-10-26T16:27:59.000Z","dependencies_parsed_at":"2024-01-12T23:18:15.307Z","dependency_job_id":"1f2f3060-3a47-4e9e-801a-e45effd831cf","html_url":"https://github.com/aidanwhiteley/books","commit_stats":{"total_commits":778,"total_committers":10,"mean_commits":77.8,"dds":0.5334190231362468,"last_synced_commit":"cfb01a7364e23495ecc0a1252e4c42a8c6fec1e8"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidanwhiteley%2Fbooks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidanwhiteley%2Fbooks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidanwhiteley%2Fbooks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aidanwhiteley%2Fbooks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aidanwhiteley","download_url":"https://codeload.github.com/aidanwhiteley/books/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332560,"owners_count":20921853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bookclub","books","docker","docker-compose","github-actions","jwt","microservice","mongo","oauth","spring-boot","spring-security","springbootadmin","swagger"],"created_at":"2024-09-24T13:13:33.757Z","updated_at":"2025-04-05T12:04:20.864Z","avatar_url":"https://github.com/aidanwhiteley.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# books\nThis project started as I wanted a simple \"microservice\" to use when trying out frameworks\nsuch as Docker, Docker Compose and Kubernetes.\n\nIt has developed a little further such that it is starting to provide some functionality that may \nactually be useful. \n\nSo welcome to the \"Cloudy Bookclub\" microservice!\n\n\u003e [!NOTE]  \n\u003e Now uplifted to the latest Spring Boot 3.x and Java 21 and with a default, HTMX based front end provided.\n\n[![Actions CI Build](https://github.com/aidanwhiteley/books/workflows/Actions%20CI%20Build/badge.svg)](https://github.com/aidanwhiteley/books/actions?query=workflow%3A%22Actions+CI+Build%22)\n[![Sonar Quality Gate](https://sonarcloud.io/api/project_badges/measure?project=com.aidanwhiteley%3Abooks\u0026metric=alert_status)](https://sonarcloud.io/dashboard?id=com.aidanwhiteley%3Abooks)\n\n## Implementation\n\nThe main functionality included in the microservice includes\n* being based on latest Spring Boot 3.x and Java 21\n* Oauth2 based logon using\n    * Google\n    * Facebook\n* the oauth2 logon data is transmogrified into locally stored users - with associated roles - and into a JWT token - \nmaking the web application entirely free of http session state (see later for whether using JWTs as session tokens is a good idea)\n* Spring Security for role based method level authorisation\n* Mongo based persistence with the use of Spring Data MongoRepository \n    * next to no persistence code\n    * except for some Mongo aggregation queries added to the Repository implementation\n* accessing the Google Books API with the Spring RestTemplate\n* and Docker images and a Docker Compose file that runs all the tiers of the application with one `docker compose up -d` command\n* new from 2025 - there is now a \"built in\" front end implementation using [HTMX](https://htmx.org/). The earlier JSON data APIs are still retained meaning that the alternative React / Typescript front end implementation still works.\n\n### Live application\nThis project runs live under Docker Compose using the HTMX front end at https://cloudybookclub.com/ and with a React / Typescript [client application](https://github.com/aidanwhiteley/books-react) available at https://react-typescript.cloudybookclub.com/\n\n![The Cloudy Book Club](../media/screengrab.jpg?raw=true)\n\n\n### Running in development\nThe checked in default Spring profile is \"mongo-java-server-no-auth\". This uses an in memory fake Mongo instance - \n[mongo-java-server](https://github.com/bwaldvogel/mongo-java-server) - so there is no need to run MongoDb locally. It also\nauto logs you on with a dummy admin user so there is no need to set up OAuth config to explore the application. So you \nshould be able to just check out the code and run and test the application for development purposes with no other dependencies. Try\n`mvnw.cmd spring-boot:run`\nand then point a browser at http://localhost:8080/\n\nTo develop Mongo related code you should switch to the \"dev\" profile which does expect to be able to connect to a real MongoDb instance.\n\nPlease check the console log when running the application. Any key constraints/warnings related to the Spring profile being used will be\noutput to the console.\n\n### Tests\nAll tests should run fine \"out of the box\" i.e. with a clean checkout of the code you should be able to successfully run\n`mvnw.cmd clean compile test package` and all tests should pass (with the code coverage reports output to target/site/jococo/index.html)\n\nBy default, the tests run against mongo-java-server so there is no need to install\nMongoDb to test most of the application. Functionality not supported by mogo-java-server such as full text indexes results in some tests being skipped when \nrunning with the mongo-java-server Spring profile.\n\nWhen running the CI builds with Github Actions, all tests run against a real Mongo instance.\n\nSome of the integration tests make use of WireMock - see the /src/test/resources/mappings and __files directories for the configuration details.\n\nThe tests are probably about 50/50 between unit tests and vastly more useful integration tests...\n\n#### Stress Test\nTo support a simple load test, there is a Maven plugin configured that runs a basic Gatling load test.\nAfter starting the Spring Boot application (i.e. mvnw.cmd spring-boot:run or via your IDE) run the command:\n\nmvnw.cmd gatling:test\n\nThe source code of this test in at test/java/com/aidanwhiteley/books/loadtest/StressTestSimulation.java. The checked in config\nensures that, by default, the number of request per second is low enough not to stress an average PC.\n\n#### Mutation Tests\nThere is support for mutation testing using the [Pitest](https://pitest.org/) library.\nTo try it out use something similar to \n`mvnw.cmd -Ppitest -DwithHistory=true -DtargetClasses=\"com.aidanwhiteley.books.service.*\" test`\nBe warned, the first run will take a long time (many minutes) - especially if the glob for targetClasses is wide. Subsequent runs should be much quicker.\nUnfortunately, this mutation support wasn't in place when the tests were originally written meaning that the \ncurrent test suite have some tests that survive too many mutations! The mutation test code is there for any new code.\n\n\n### How to build and run\nThis project makes use of the excellent Lombok project. So to build in your favourite IDE, if necessary\nhead on over to [Lombok](https://projectlombok.org/) and click the appropriate \"Install\" link (tested with IntelliJ and Eclipse).\n\nIn preparation for playing with recent Java features such as virtual threads and pattern matching, the build of this project **now requires JDK21**.\n\nWith appropriate versions of the JDK (i.e. 21+) and optionally Maven and Mongo installed, start with\n~~~~\nmvnw.cmd clean compile test\n~~~~\nand then try\n~~~~\nmvnw.cmd spring-boot:run\n~~~~\nTo access the application using the default HTMX based front end, point your browser to http://localhost:8080/\n\nTo run a client Single Page Application to access the microservice API, head on over to https://github.com/aidanwhiteley/books-react or see the section below on using Docker.\n\n### Available Spring profiles\nThere are Spring profile files for a range of development and test scenarios. \n\n#### dev-mongo-java-server-no-auth (the default selected in application.yml)\n\t- uses an in memory mongo-java-server rather than a real MongoDb\n\t- configured such that all request have admin access. No oauth set up required and no logon\n\t- clears down the DB and reloads test data on every restart\n\n#### dev-mongo-java-server \n\t- uses an in memory mongo-java-server rather than a real MongoDb\n\t- requires oauth configured correctly for access to update operations\n\t- clears down the DB and reloads test data on every restart\n\t\n#### dev-mongodb-no-auth\n\t- uses a real MongoDb\n\t- configured such that all request have admin access. No oauth set up required and no logon\n\t- clears down the DB and reloads test data on every restart\n\t\n#### dev-mongodb\n\t- uses a real MongoDb\n\t- requires oauth configured correctly for access to update operations\n\t- clears down DB and reloads test data on every restart\n\t\n#### CI\n\t- uses a real MongoDb\n\t- clears down the DB and reloads test data on every restart\n\t\n#### container-demo-no-auth\n    - requires the use of \"docker-compose up\" to start Docker containers - see later\n\t- uses a real MongoDb\n\t- configured such that all request have admin access and oauth config is not required\n\t- clears down the Mongo DB and reloads test data on every restart of the Docker containers\n\t\n\n### Configuring for production\n\"Out of the box\" the code runs with the \"mongo-java-server-no-auth\" Spring profile - see the first line of application.yml. None\nof the checked in available Spring profiles are intended for production use. You will need to decide the \nrequired functionality for your environment and configure your Spring profile accordingly. \n\nFor instance, you **WILL** want to \nset/change the secretKey used for the JWT token signing (see books:jwt:secretKey in the yml files).\nPlease see the code in com.aidanwhiteley.books.controller.jwt.JwtUtils::createRandomBase64EncodedSecretKey\nwhich will allow you to generate a random SecretKey rather than using the default one supplied\nin the non-propduction properties files.\n\nYou will also need access to a Mongo instance. The connection URL (in the yml files) will result in the automatic\ncreation of a Mongo database and the two required collections (dependant on the security config of your Mongo install).\n\nCheck the console log when running in production - you should see **NO** warning messages!\n\n### How to configure application logon security\nA lot of the functionality is normally protected behind oauth2 authentication (via Google and Facebook).\nTo use this, you must set up credentials (oauth2 client ids) on Google and/or Facebook.\nYou must then make the client-id and client-secret available to the running code.\nThere are \"placeholders\" for these in /src/main/resources/application.yml i.e. replace the existing\n\"NotInSCMx\" (Not In Source Code Management!) values with your own.\n\n### Sample data\nThere is some sample data provided to make initial understanding of the functionality a bit easier.\nIt is in the /src/main/resources/sample_data. See the #README.txt in that directory.\nSee the details above for the available Spring profiles to see when this sample data is autoloaded.\n\n#### Indexes\nThe Mongo indexes required by the application are not \"auto created\" (except when running in Docker containers).\nYou can manually apply the indexes defined in /src/main/resources/indexes.\nIn particular, the application's Search functionality won't work unless you run the command to build\nthe weighted full text index across various fields of the Book collection. The rest of the application will run without \nindexes - just more slowly as the data volumes increase!\n\n#### Admin emails\nThere is functionality to send an email to an admin when a new user has logged on. This is intended to prompt the\nadmin to give the new user the ROLE_EDITOR role (or delete the user!).\nThis functionality must be enabled - see the books.users.registrationAdminEmail entries in application.yml (where \nit is disabled by default).\n\n## Levels of access\nThe code supports five access levels\n* anonymous (never logged in)\n* ROLE_USER (logged in but no more permissions than anonymous)\n* ROLE_EDITOR (logged in with permission to create book reviews and comment on other people's book reviews)\n* ROLE_ADMIN (logged in with full admin access)\n* ROLE_ACTUATOR (logged in but with no permissions except to access Actuator endpoints)\n\nThe application-\u003cprofile\u003e.yml files can be edited to automatically give a logged on user admin access \nby specifying their email on Google / Facebook. See the books:users:default:admin:email setting.\n\n## Security\nThis demo application stores a JWT token in a secure and \"httpOnly\" cookie. So that hopefully blocks some XSS exploits \n(as the rogue JavaScript can't read the httpOnly cookie containing the JWT logon token). \n\nTo mitigate XSRF exploits, the application uses an XSRF filter to set an XSRF token in a (non httpOnly) cookie that must be\nre-sent to the server for state changing (non GET) requests as an X-XSRF-TOKEN request header. The filter checks that the two values are the same.\n\n**Note** - as of early 2025, this application no longer supports setting CORS headers to allow the front end to be on a different domain to the API. You should avoid this by using a reverse proxy or API gateway. For SPA development, a middleware proxy is recommended (see https://github.com/aidanwhiteley/books-react for an example)\n\n### HTMX security details\nThe included Thymeleaf / HTMX application follows the HTMX best practises detailed at https://htmx.org/docs/#security. In addition\n* `htmx.config.allowEval` is set to false to prevent the HTMX's use of JavaScript's `eval()`\n* While mostly agreeing with the concept of [Locality of Behaviour](https://htmx.org/essays/locality-of-behaviour/), all JavaScript has been removed from the Thymeleaf templates to separate JS files. This is to allow a usefully strict Content Security Policy to be set\n\n## OpenAPI 3 API documentation and swagger-ui\n\n[![API Documentation](https://github.com/aidanwhiteley/books/blob/develop/src/main/resources/static/swagger-logo.png)](https://cloudybookclub.com/swagger-ui/index.html)\n\nThe public read only part of the application's JSON API is automatically documented using the [springdoc-openapi](https://springdoc.org/)\ntool to auto create OpenAPI 3 JSON. The API documentation can be explored and tested using the Swagger UI available [here](https://cloudybookclub.com/swagger-ui/index.html).\n\n## Stateless Apps\nA lot of the time developing this microservice was spent in making it entirely independent of HTTP session state  - based around issuing a \nJWT after the user has authenticated via Google / Facebook.\n\nThis turned out to be surprisingly difficult - with the cause of the difficulty mainly being in the Spring Boot OAuth2 implementation \nin Spring Boot 1.x. The Google/Facebook re-direct back to the microservice needed to hit the same session / JVM as I think that the \nOauth2ClientContext was storing some state in http session. \n\nThe current version of this application has moved to the Oauth2 functionality in Spring Security 6. While this greatly reduces the \n\"boilerplate\" code needed to logon via Google or Facebook it still, by default, stores data in the HTTP session (to validate the data in the redirect back from Google / Facebook).\nHowever, it does allow configuration of your own AuthorizationRequestRepository meaning that it is possible to implement a cookie\nbased version. So, finally, this application is completely free of any HTTP session state! Which was the point of originally starting to write this \nmicroservice...\n\n### Using JWT for session management\nThis demo app uses JWTs as the user's session token. In most cases this is now considered an \nanti-pattern ([pros and cons discussion](https://news.ycombinator.com/item?id=27136539)).\nFor my own usage (where I'm extremely unlikely to need to quickly revoke users) it's fine but I wouldn't use this solution \nfor user session management again in the future.\n\n\u003e [!WARNING]  \n\u003e There was a bug in versions of this project prior to those using Spring Boot 3 i.e. \n\u003e prior to 0.30.0-SNAPSHOT in that random plaintext was being passed to the JWT signing\n\u003e function rather than a SecretKey. Please see [this StackOverflow discussion](https://stackoverflow.com/questions/40252903/static-secret-as-byte-key-or-string/40274325#40274325) for more details.\n\n## Docker\nDocker images are available for the various tiers that make up the full application.\n### Docker web tier\nAn nginx based Docker image (aidanwhiteley/books-web-react) is available that hosts the React/Typescript single page app\nand acts as the reverse proxy through to the API tier (this application). See https://github.com/aidanwhiteley/books-react\nfor more details.\nThe checked in docker-compose.yml specifies the use of aidanwhiteley/books-web-react-gateway which routes Ajax\ncalls to the APIs via a Spring Cloud Gateway based API gateway.\n### API Gateway\nA Spring Cloud Gateway based API gateway image is available to route web traffic on port 8000 through to \n(a cluster of) Spring Boot based books microservices.\nAlso provides basic throttling and load balancing amongst the available books microservices.\nRequires a Service Registry to register with and find running books microservices.\n### Service Registry \nA Spring / Netflix Eureka service registry in which instances of the books microservice register themselves and in\nwhich the API gateway finds available instances of the books microservice.\n### Java API tier\nThere is Google Jib created image (aidanwhiteley/books-api-java) for this application. \nThe image can be recreated by running \"mvnw.cmd compile jib:dockerBuild\".\nRegisters with Service Registry (dependant on the Spring profile used).\n### MongoDB data tier\nA MongoDB based Docker image (aidanwhiteley/books-db-mongodb or aidanwhiteley/books-db-mongodb-demodata) is available\nto provide data tier required by this application.\nUse the aidanwhiteley/books-db-mongodb-demodata to have sample data reloaded into the MongoDB every time the \ncontainer is restarted.\nSee the src/main/resources/mongo-docker directory for Docker build of the data tier.\n## Docker compose\nThere is a docker-compose.yml file in the root of this application. This starts Docker containers for all the above \ntiers of the overall application.\n### .env file\nThe docker-compose file expects there to be a .env file in the same directory to define the environment \nvariables expected by the various Docker images.\nThere is an example .env file with comments checked in. This **SHOULD** be edited according to the \ninstructions in the file (noting that the containers will start and run OK with the checked in values if you are just trying things out).\n\n## Docker Compose and running the overall application\nThe checked in docker-compose.yaml and .env file should result in a deployment as shown on the following diagram.\n\n![Cloudy Docker Deployment Diagram](../media/docker1.png?raw=true)\n\nIf you have a recentish Docker available, from the root of this project type:\n~~~~\ndocker compose pull\ndocker compose up -d --scale api-tier-java=2\n~~~~\n\nThen try accessing http://localhost/ \n\nYou may get 503 errors for a short period while the whole stack is starting up.\n\nNote: If you want to persist data in Mongo between restarts of the container, rename the file docker-compose.override.yaml.persistent-data to docker-compose.override.yaml\nIf you do this and are running on Windows, make sure to read the Caveats section of https://hub.docker.com/_/mongo/\n\n|Tier |URL | Notes and screen grab                                                                                                                                                                                                                                                                            |\n|-----|----|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n|Website|http://localhost/| Accessing this URL, if you left the docker-compose.yaml as is, you will be auto \"logged on\" as an admin user. Try the \"ADD A BOOK\" link and, after you have completed the \"Title\", you should see a list of matching books from Google Books.![Books web tier](../media/screengrab.jpg?raw=true) |\n|Service registry|http://localhost:8761/| The service registry isn't behind logon. You should see registered the API gateway, the service registry itself, two instances of the Books microservice and a Spring Boot Admin instance.![Books service registry](../media/service-registry.jpg?raw=true)                                      |\n|Spring Boot Admin|http://localhost:8888/| If you haven't changed the .env file, logon with anAdminUser-CHANGE_ME / aPassword-CHANGE_ME ![Books Spring Boot Admion](../media/spring-boot-admin.jpg?raw=true)                                                                                                                                |\n|Mongo data tier|http://localhost:27017/| With the default docker-compose.yml this isn't exposed. Look for the comments in the file around about line 136 if you want to access the database directly (userids and passwords being in the .env file) ![Mongo](../media/mongo-compass.jpg?raw=true)                                         |\n\n\n## Spring Boot Admin\nThe application supports exposing [Actuator](https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready.html) \nendpoints to be consumed by [Spring Boot Admin](http://codecentric.github.io/spring-boot-admin/current/). We need security applied to \nthe Actuator end points but don't want to introduce another security layer into the application - we want to stick with the JWT based implemetation \nwe already have. So we need Spring Boot Admin to be able to supply a suitable JWT token when calling the Actuator end points. \n\nTo do this, set books.allow.actuator.user.creation to true and run the application. A JWT will be printed to the application logs for a \nuser that **only** jas the Actuator role. Plug this JWT token into a Spring Boot Admin application that is configured to send the above JWT token with each \nrequest to the Actuator endpoints in this application. A extract of the required configuration of a class that\nimplements de.codecentric.boot.admin.server.web.client.HttpHeadersProvider is listed below \nwith a fully working example project being available at https://github.com/aidanwhiteley/books-springbootadmin\n```java\n@Component\npublic class JwtHeaderProvider implements HttpHeadersProvider {\n    \n    @Override\n    public HttpHeaders getHeaders(Instance instance) {\n        HttpHeaders headers = new HttpHeaders();\n        headers.add(HttpHeaders.COOKIE, JWT_COOKIE_NAME + \"=\" + jwtTokenActuatorUser);\n        return headers;\n    }\n}\n```\n* Set HTTP basic username/password values required when the client application registers with the Spring Boot Admin instance\n    * in the client application (i.e. this application) by setting the spring.boot.admin.client.username/password values \n    * configure the Spring Boot admin application with the same values by setting spring.security.user.name/password\n\n## The name\n\nWhy \"The Cloudy BookClub\"? Well - it's going to run in the cloud innit. And I couldn't think\nof any other domain names that weren't already taken.\n\n## Client Side Functionality\n\nThere's a choice of two front end implementations.\n\nA Thymeleaf / HTMX based Multiple Page Application implementation is included in this project (see the src/main/resources/templates directory for the Thymeleaf templates and HTMX code). This implementation can be seen running at https://cloudybookclub.com/\n\nThere is a React/Typescript based Single Page Application front end application that consumes the microservice is available at https://github.com/aidanwhiteley/books-react. This implementation can be seen running at https://cloudybookclub.com/react-typescript\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faidanwhiteley%2Fbooks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faidanwhiteley%2Fbooks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faidanwhiteley%2Fbooks/lists"}