{"id":22177604,"url":"https://github.com/ntidev/syncbundle","last_synced_at":"2025-10-08T19:42:09.463Z","repository":{"id":46787993,"uuid":"112134319","full_name":"ntidev/SyncBundle","owner":"ntidev","description":"Synchronization bundle","archived":false,"fork":false,"pushed_at":"2023-07-19T19:32:53.000Z","size":243,"stargazers_count":3,"open_issues_count":1,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-07-08T14:01:42.296Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ntidev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-11-27T01:54:47.000Z","updated_at":"2023-02-14T15:17:59.000Z","dependencies_parsed_at":"2024-12-02T08:41:54.343Z","dependency_job_id":null,"html_url":"https://github.com/ntidev/SyncBundle","commit_stats":{"total_commits":67,"total_committers":7,"mean_commits":9.571428571428571,"dds":"0.31343283582089554","last_synced_commit":"077d4b053674b7935378cbd29da102583259d1a5"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/ntidev/SyncBundle","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntidev%2FSyncBundle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntidev%2FSyncBundle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntidev%2FSyncBundle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntidev%2FSyncBundle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ntidev","download_url":"https://codeload.github.com/ntidev/SyncBundle/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntidev%2FSyncBundle/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279000703,"owners_count":26082806,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-02T08:29:54.201Z","updated_at":"2025-10-08T19:42:09.421Z","avatar_url":"https://github.com/ntidev.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NTISyncBundle\n\n\n### Installation\n\n1. Install the bundle using composer:\n\n    ```\n    $ composer require nti/sync-bundle \"dev-master\"\n    ```\n\n2. Add the bundle configuration to the AppKernel\n\n    ```\n    public function registerBundles()\n    {\n        $bundles = array(\n            ...\n            new NTI\\SyncBundle\\NTISyncBundle(),\n            ...\n        );\n    }\n    ```\n\n3. Update the database schema\n\n    ```\n    $ php app/console doctrine:schema:update\n    ```\n\n4.  Add the routes to your `routing.yml`\n\n    ```\n    ...\n    nti_sync:\n        resource: \"@NTISyncBundle/Resources/config/routing.yml\"\n    ```\n\n## Requirements\n\nBelow are a list of things that need to be considered in order to implement this bundle:\n\n1. Any Entity that needs to be taken into account during the synchronization process must have the `@NTI\\SyncEntity` annotation at the class level.\n2. `ManyToOne` relationships that should alter the last synchronization timestamp of their parents should use the annotation `@NTI\\SyncParent(getter=\"[Getter Name]\")` (see example below for more information).\n3. Entities to be synced must have a repository implementing the `SyncRepositoryInterface` (see below for more information).\n4. The mapping `SyncMapping` needs to be configured foreach entity as it is the list used as reference for the lookup.\n5. The `SyncState` should be created for each mapping. This can be done with this query after creating all the `SyncMapping`:\n    ```\n    `INSERT INTO nti_sync_state(mapping_id, timestamp) SELECT id, 0  FROM sync_nti_mapping;`\n6. If the entity is going to be synched FROM the client, then a service must be defined in the `SyncMapping` database entry. Also, this method needs to implement the interface `SyncServiceInterface`. \n\n## Tracking Changes\n\nThe way that the bundle tracks changes in the synchronization is as follows:\n\n1. The bundle has a `DoctrineEventListener` listening to the `onFlush` event.\n2. Once the event is fired, the bundle will grab every entitty that has the `@NTI\\SyncEntity` annotation.\n3. If the entity has a `SyncMapping` defined, the system will update the `last_timestamp` field of this mapping to the current `time()`.\n4. If the entity has a method called `setLastTimestamp()` it will be called with the `time()` as a parameter and the changes will be recomputed or computed.\n5. All the properties of the entity will be examined in search for a property that contains the annotation `@NTI\\SyncParent(getter=\"[Getter Name]\")`.\n   If found, the getter will be called, if the result is an object that also has the `@NTI\\SyncEntity`, it will be processed again starting from point #3. This process occurrs recursively.\n  \n## Configuration\n\nBelow is the default configuration for the bundle. In case you need to modify the defaults this would go in your `config.yml`:\n\n```yaml\nnti_sync:\n  deletes:\n      \n    # Identifier to use when an item gets deleted. This would go in your `deletes` section as shown below\n    identifier_getter: \"getId\"\n        \n\n```\n\n## Class Examples\n\n    ```\n    \u003c?php\n    \n    ...\n    use NTI\\Annotations as NTI;    \n    \n    /**\n     * ...\n     * @NTI\\SyncEntity()\n     */\n    public class Product {\n        ...        \n        \n        /**\n         * @ORM\\Column(name=\"last_timestamp\", type=\"bigint\", options={\"default\": 0})\n         */\n        private $lastTimestamp;\n        \n        ...\n        \n        /**\n         * Set lastTimestamp\n         * @param $lastTimestamp\n         * @return Company\n         */\n        public function setLastTimestamp($lastTimestamp)\n        {\n            $this-\u003elastTimestamp = $lastTimestamp;    \n            return $this;\n        }\n    \n        /**\n         * Get lastTimestamp\n         * @return integer\n         */\n        public function getLastTimestamp()\n        {\n            return $this-\u003elastTimestamp;\n        }\n\n    }    \n    \nAn example of a class using a `ManyToOne` where the child also needs the parent's `last_timestamp` to be updated can be defined as:\n\n    ```\n    \u003c?php\n    \n    ...\n    use NTI\\Annotations as NTI;    \n    \n    /**\n     * ...\n     * @NTI\\SyncEntity()\n     */\n    public class ProductChild {\n        ...        \n                \n        /**\n         * @NTI\\SyncParent(getter=\"getProduct\")\n         * @ORM\\ManyToOne(targetEntity=\"AppBundle\\Entity\\Product\\Product\")\n         */\n        private $product;\n        \n        ...\n            \n        /**\n         * Get Product\n         * @return Product\n         */\n        public function getProduct()\n        {\n            return $this-\u003eproduct;\n        }\n    }          \n  \nBelow is the general process that the bundles goes through to keep track of the synchronization state:\n\n![Synchronization Process - Server](/Images/SynchronizationProcess-Server.PNG?raw=true \"Synchronization State Process on the Server\")\n\nBelow is the general process that occurs when a client asks for the changes after a specific timestamp:\n\n![Synchronization Process - Client](/Images/SynchronizationProcess-Client.PNG?raw=true \"Synchronization Process on the Client\")\n\n## Implementation (Pull)\n\nThe idea behind the synchronization process is that every object that is going to be synchronized should implement the `SyncRepositoryInterface` in its repository.\n\n```\n/**\n * Interface SyncRepositoryInterface\n * @package NTI\\SyncBundle\\Interfaces\n */\ninterface SyncRepositoryInterface {\n    /**\n     * This function should return a plain array containing the results to be sent to the client\n     * when a sync is requested. The container is also passed as a parameter in order to give additional\n     * flexibility to the repository when making decision on what to show to the client. For example, if the user\n     * making the request only has access to a portion of the data, this can be handled via the container in this method\n     * of the repository.\n     *\n     * Note 1: If the `updatedOn`  of a child entity is the one that is affected and not the parent, you may have to take that\n     *         into account when doing your queries so that the updated information shows up in the results if desired when doing\n     *         the comparison with the timestamp\n     * \n     *         For example:\n     *         \n     *              $qb -\u003e ...\n     *              $qb -\u003e leftJoin('a.b', 'b')\n     *              $qb -\u003e andWhere($qb-\u003eexpr()-\u003eorX(\n     *                  $qb-\u003eexpr()-\u003egte('a.lastTimestamp', $date),\n     *                  $qb-\u003eexpr()-\u003egte('b.lastTimestamp', $date)\n     *              ))\n     *              ...\n     *              \n     *         This way if the only way of syncronizing B is through A, next time A gets synched B changes will be reflected. \n     * \n     * The resulting structure should be the following:\n     * \n     * array(\n     *      \"data\" =\u003e (array of objects),\n     *      SyncState::REAL_LAST_TIMESTAMP =\u003e (last updated_on date from the array of objects),\n     * )\n     *     \n     *\n     * @param $timestamp\n     * @param ContainerInterface $container\n     * @param array $serializationGroups\n     * @return mixed\n     */\n    public function findFromTimestamp($timestamp, ContainerInterface $container, $serializationGroups = array());\n```\n\nBesides implementing the interface, in the database `nti_sync_mapping` the mapping for each class that is going to be synchronized should be configured along with a name.\n\n\nFirst, the idea is to get a summary of the changes and mappings from the server:\n\n```\nGET /nti/sync/summary\n```\n\nTo which the server will respond with the following structure:\n\n```\n[\n    {\n        \"id\": 1,\n        \"mapping\": {\n            \"id\": 1,\n            \"name\": \"Product\",\n            \"class\": \"AppBundle\\\\Entity\\\\Product\\\\Product\",\n            \"sync_service\": \"AppBundle\\\\Service\\\\Product\\\\ProductService\"\n        },\n        \"timestamp\": 1515076764\n    },\n    ...\n]\n```\n\nThe response contains a list of mappings with their last registered timestamp. This timestamp can be used in the synchronization process to figure out what has changed and what needs to be synced.\n\n\nThen, a third party makes a request to the server using the following structure:\n\n```\nPOST /nti/sync/pull\nContent-Type: application/json\n{\n    \"mappings\": [\n        { \"mapping\": \"[MAPPING_NAME]\", \"timestamp\": [LAST_TIMESTAMP_CHECKED] }\n    ]\n}\n```\n\nAfter receiving the request, if a mapping with the specified name exists, the system will call the repository's findFromTimestamp implementation and return the following result (Using a Product entity as an example):\n\n```\n{\n    \"[MAPPING_NAME]\": {\n        \"changes\": [\n            {\n                \"id\": 2,\n                \"productId\": \"POTATOBAG\",\n                \"name\": \"Potato bag\",\n                \"description\": \"Bag of potatoes\",\n                \"price\": \"32.99\",\n                \"cost\": \"0\",\n                \"createdOn\": \"11/30/2017 04:22:49 PM\",\n                \"updatedOn\": \"11/30/2017 04:22:49 PM\",\n                \"lastTimestamp\": 1515068439\n            },\n            ...\n        ],\n        \"newItems\": [\n            {\n                \"id\": 1,\n                \"uuid\": \"24a7aff0-fea8-4f62-b421-6f97f464f310\",\n                \"mapping\": {\n                    \"id\": 1,\n                    \"name\": \"Product\",\n                    \"class\": \"AppBundle\\\\Entity\\\\Product\\\\Product\",\n                    \"sync_service\": \"AppBundle\\\\Service\\\\Product\\\\ProductService\"\n                },\n                \"class_id\": 8,\n                \"timestamp\": 1515068439\n            },\n            ...            \n        ],\n        \"deletes\": [\n            {\n                \"id\": 2,\n                \"mapping\": {\n                    \"id\": 2,\n                    \"name\": \"Product\",\n                    \"class\": \"AppBundle\\\\Entity\\\\Product\\\\Product\"\n                },\n                \"classId\": \"[identifier_getter result]\",\n                \"timestamp\": 1512080746\n            },\n            ...\n        ],\n        \"failedItems\": [\n            {\n                \"id\": 7,\n                \"uuid\": \"abcdefg-123456-hifgxyz-78901\",\n                \"mapping\": {\n                    \"id\": 9,\n                    \"name\": \"Product\",\n                    \"class\": \"AppBundle\\\\Entity\\\\Product\\\\Product\",\n                    \"sync_service\": \" ... \"\n                },\n                \"classId\": 137,\n                \"timestamp\": 1512080747,\n                \"errors\": [...errors provided...]\n            },\n            ...\n        ],\n        \"_real_last_timestamp\": 1512092445\n    }\n}\n\n```\n\nThe server will return the both the `changes` , `newItems`, `failedItems` , and the `deletes`. The `changes` will contain the `data` portion of the array returned by\nthe repository's implementation of `SyncRepositoryInterface`. The `deletes` will contain the list of `SyncDeleteState` that were recorded since the \nspecified timestamp. The `newItems` will contain the list of `SyncNewItemState` which means the new items that were created since the provided timestamp\nincluding the UUID that was given at the time (This is helpful to third party devices when first pulling the information they can verify if an item was already created\nbut they don't have the ID of that item in their local storage and avoid creating duplicates in the server). The `failedItems` will contain the list of `SyncFailedItemState`, each item in this list\ncontains an `errors` property with the errors founds processing the creation or update of the entity. \n\nThe `_real_last_timestamp` should be used as it can help with paginating the results for a full-sync and help the client\nget the real last timestamp of the last object in the response. This has to be obtained in the repository and can be done\nby simply getting the last item from the repository's result and calling the `getLastTimestamp()`.\n\nFrom this point on, the client must keep a track of the `_real_last_timestamp` in order to perform a sync in the future.\n\n## Implementation (Push)\n\nBelow is the general idea over the push/pull process:\n\n![Synchronization Process - Push/Pull](/Images/SynchronizationProcess-PushPull.PNG?raw=true \"Synchronization Push Pull Process\")\n\n### Server Side\nIn the `SyncMapping` for each mapped entity a service should be specified. This service must implement the `SyncServiceInterface`. \n\n### Client Side\nIn order to handle a push from a third party device it must provide the following structure in its request:\n \n```\nPOST /nti/sync/push\nContent-Type: application/json\n{\n    \"mappings\": [\n        { \"mapping\": \"[MAPPING_NAME]\", \"data\": [\n            {\n                \"id\": \"5eb86d4a-9b82-42f3-abae-82b1b61ad58e\",\n                \"serverId\": 1,\n                \"name\": \"Product1\",\n                \"description\": \"Description of the product\",\n                \"price\": 55,\n                \"lastTimestamp\": 1512080746\n            },\n            ...\n        ] }\n    ]\n}\n```\n\nWhen the server receives this request, it will execute the `sync()` method of the configured service in the respective `SyncMapping` and it will\npass the array of data for that parameter. Your `sync()` function needs to operate over this information and return an array which will be included\ninside the response under the respective mapping name.\n\nThe server then returns the following structure:\n```\n{\n    \"mappings\": [\n        { \"[MAPPING_NAME]\": \"RESULT OF YOUR sync() HERE\" },\n        { \"[MAPPING_NAME]\": \"RESULT OF YOUR sync() HERE\" },\n        { \"[MAPPING_NAME]\": \"RESULT OF YOUR sync() HERE\" },\n    ]\n}\n```\n\n## Todo\n\n* Handle deletes from third parties\n* `ManyToMany` relationships are tricky and can lead to performance issues \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntidev%2Fsyncbundle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fntidev%2Fsyncbundle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntidev%2Fsyncbundle/lists"}