{"id":23182757,"url":"https://github.com/chenxyzl/mongoincupdate.fody","last_synced_at":"2026-04-28T13:37:41.136Z","repository":{"id":225311224,"uuid":"558326248","full_name":"chenxyzl/MongoIncUpdate.Fody","owner":"chenxyzl","description":"mongo Incremental update(include dictionary)","archived":false,"fork":false,"pushed_at":"2022-11-21T02:58:16.000Z","size":202,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-05T03:24:02.464Z","etag":null,"topics":["incremental-update","mongo"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chenxyzl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"License.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-10-27T10:25:04.000Z","updated_at":"2024-03-01T09:12:06.000Z","dependencies_parsed_at":"2024-03-01T11:31:11.737Z","dependency_job_id":"516c3760-44fa-44f9-9ce8-590187f25576","html_url":"https://github.com/chenxyzl/MongoIncUpdate.Fody","commit_stats":null,"previous_names":["chenxyzl/mongoincupdate.fody"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/chenxyzl/MongoIncUpdate.Fody","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenxyzl%2FMongoIncUpdate.Fody","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenxyzl%2FMongoIncUpdate.Fody/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenxyzl%2FMongoIncUpdate.Fody/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenxyzl%2FMongoIncUpdate.Fody/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chenxyzl","download_url":"https://codeload.github.com/chenxyzl/MongoIncUpdate.Fody/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chenxyzl%2FMongoIncUpdate.Fody/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261272008,"owners_count":23133784,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["incremental-update","mongo"],"created_at":"2024-12-18T08:21:57.103Z","updated_at":"2026-04-28T13:37:36.113Z","avatar_url":"https://github.com/chenxyzl.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PropertyChangeWatch.Fody\n# mongo增量方案\n1. 给所有属性在编译器静态注入脏标记\n2. 集合类型:Dictionary用StateMap替换,暂不支持List(会退化为全量保存).\n3. mongo存储时候检查藏标记来生成UpdateDefinition,并执行UpdateOneAsync保存\n\n## 实现原理(如不关心直接看底部如何使用)\n### 实现增量更新的脏标记注入\n1. 引用Fody包,增加FodyWeavers.xml,配置导入MongoIncUpdate(MongoIncUpdate工程为Fody的静态代码编织插件,增量方案的代码注入在这里实现)\n2. MongoIncUpdate工程的增加类ModuleWeaver(继承BaseModuleWeaver),会被fody调用(原理就是msbuild会在编译以前扫描引入的包的.targets文件,并执行其中的task,详情看《07_.net fody.md》)\n    1. ModuleWeave实现扫描标记为MongoIncUpdateAttribute为等待被注入的对象\n    2. ModuleWeave实现扫描标记为MongoIncUpdateInterfaceAttribute为被注入的接口(基于插件模型)\n    3. 对每个MongoIncUpdateAttribute标记的对象分别注入MongoIncUpdateInterfaceAttribute标记的接口\n    4. 对每个MongoIncUpdateAttribute标记的对象分别注入接口的属性(Dirties,IdxMapping,NameMapping)实现\n    5. 对每个MongoIncUpdateAttribute标记的对象分别注入_dirties,_idxMapping,_nameMapping字段\n    6. 对每个MongoIncUpdateAttribute标记的对象分别注入set/get_Dirties,set/get_IdxMapping,set/get_NameMapping字段\n    7. 对上述注入的setter和getter分别注入消息体,并在setter中注入调用MongoIncUpdateInterfaceAttribute标记的插件类的静态方法PropChange。实现了属性变化通知\n    8. 扫描对象的所有ctor方法,注入调用MongoIncUpdateInterfaceAttribute标记的插件类的实例方法Init\n3. 实现StateMap和StateMapSerializer增加对Dictionary的支持\n\n### Mongo增量的存储过程\n1. 存储对象调用MongoIncUpdateInterfaceAttribute注入的接口的BuildUpdate来生成UpdateDefinition\n2. BuildUpdate中for循环遍历脏标记,更具标记的位置来获取对象的值和类型相关属性\n3. 如果是脏则整体存储,如果不是脏则检查是否是引用类型(string特殊引用类型除外),如果类型没有被注入插件接口则退化为整体存储(保底措施),有递归调用BuildUpdate\n4. 增加StateMap(继承至MongoIncUpdateInterfaceAttribute标记的接口)来支持集合类型,(实现了StateMapSerializer来支持序列化反序列化)   \n   这里注意dic[key]=value时候,mongo的序列化有个大坑\n   1. 在StringFieldDefinitionHelper.Resolve获取类型时候对于正常的obj.property = value中, value的类型是来至于obj的member中同名property的类型获取出来的。\n   2. 在obj[key]=value中。也会变为obj.key=value形式,走上述类型推断逻辑,会变成获取obj中成员同名为key的类型,而字典的key最终都为string类型。接下来又两个错误   \n        1. key为全部数值类型,尝试转化为IBsonArraySerializer且失败,直接跳出类型推断代码,在上层退化为默认类型。\n        2. 非全数值类型,尝试转化为IBsonDocumentSerializer类型,而key一般为string类型,转换失败,直接跳出类型推断代码,在上层退化为默认类型。\n   3. 解决方法\n        1. StateMapSerializer的序列化时候把key转换为k_key的形式,避免上述中全数值类型被终止类型插件   \n        2. 接着实现IBsonDocumentSerializer接口,在TryGetMemberSerializationInfo函数中返回value的序列化器。key的序列化器是StateMapSerializer\u003cTKey, TValue\u003e,不会打断递归,且key会被检查继承关系,检查失败会退化为原生类型,能保证key正常序列化。对于value:此操作类似把value当作了key的成员类型。\n5. 调用await collection.UpdateOneAsync(filter, setter, new UpdateOptions { IsUpsert = true });来保存\n\n## 实现过程的伪代码\n``` c#\npublic sealed NeedInject //: IDiffUpdateable  //0.注入接口IDiffUpdateable\n{\n    //1.构造函数注入调用IDiffUpdateable.Init()\n    // NeedInject(){\n    //     //构造函数注入调用IDiffUpdateable.Init()\n    // }\n    //2.注入属性Dirties\n    //覆盖属性 IDiffUpdateable.Dirties\n    //private static readonly BitArray _dirties = new(0);\n    //BitArray IDiffUpdateable.Dirties { get; set; } = _dirties;\n    //3.注入静态成员IsOnceInitDone\n    //覆盖 IDiffUpdateable.IsOnceInitDone\n    // private static readonly bool _isOnceInitDone = false; //跳过初始化逻辑\n    // bool IDiffUpdateable.IsOnceInitDone { get; set; } = _isOnceInitDone;\n    //4.注入静态成员NameMapping\n    //覆盖 IDiffUpdateable.NameMapping\n    // private static readonly Dictionary\u003cstring, int\u003e _nameMapping = new();\n    // Dictionary\u003cstring, int\u003e IDiffUpdateable.NameMapping { get; set; } = _nameMapping;\n    //5.注入静态成员NameMapping\n    //覆盖 IDiffUpdateable.IdxMapping\n    // private static readonly Dictionary\u003cint, IPropertyCallAdapter\u003e _idxMapping = new();\n    // Dictionary\u003cint, IPropertyCallAdapter\u003e IDiffUpdateable.IdxMapping { get; set; } = _idxMapping;\n    public int PropA { get; set;} //注入setter调用IDiffUpdateable.PropChange\n}\n```\n\n## 如何使用\n### 1.\u003cfont color=red\u003e注册序列化器\u003c/font\u003e(因为嵌套类型需要)\n最好在链接mongo数据库之前   \n\u003cfont color=red\u003eBsonSerializer.RegisterGenericSerializerDefinition(typeof(StateMap\u003c,\u003e), typeof(StateMapSerializer\u003c,\u003e));\u003c/font\u003e\n\n### 2.存储实体构造\n``` C#\n[MongoIncUpdate]\npublic class Inner2\n{\n    //多层嵌套 任意测试了  \n    public int I { get; set; }\n    [BsonIgnore]\n    public int XX { get; set; }\n}\n\n[MongoIncUpdate]\npublic class Inner1\n{\n    //测试嵌套的dictionary的引用类型嵌套\n    [BsonSerializer(typeof(StateMapSerializer\u003cstring, Inner2\u003e))]\n    public StateMap\u003cstring, Inner2\u003e Dic1 { get; set; } = new();\n    \n    public Inner2 Inner2 { get; set; } = new();\n}\n\n[MongoIncUpdate]\npublic class Item\n{\n    //id\n    [BsonId] public int Id { get; set; }\n\n    //string类型带attr\n    [BsonElement(\"RealName\")] public string Name { get; set; }\n    //\n    //测试引用类型\n    public Inner1 Inner1 { get; set; } = new();\n    //\n    // //测试dictionary的值类型\n    [BsonSerializer(typeof(StateMapSerializer\u003cint, int\u003e))]\n    public StateMap\u003cint, int\u003e Dic1 { get; set; } = new();\n\n    //测试dictionary的引用类型嵌套\n    [BsonSerializer(typeof(StateMapSerializer\u003cstring, Inner1\u003e))]\n    public StateMap\u003cstring, Inner1\u003e Dic2 { get; set; } = new();\n}\n```\n\n### 3.保存数据\n```c#\npublic static async Task SaveIm(this Item self, IMongoCollection\u003cItem\u003e collection)\n    {\n        var diffUpdateable = self as IDiffUpdateable;\n        var defs = new List\u003cUpdateDefinition\u003cItem\u003e\u003e();\n        diffUpdateable?.BuildUpdate(defs, \"\");\n        if (defs.Count == 0) return;\n        var setter = Builders\u003cItem\u003e.Update.Combine(defs);\n        var filter = Builders\u003cItem\u003e.Filter.Eq(\"_id\", self.Id);\n        await collection.UpdateOneAsync(filter, setter, new UpdateOptions { IsUpsert = true });\n        Console.WriteLine($\"update data count:{defs.Count}\");\n    }\n```\n\n## 性能测试\nBenchmark.md[链接](./Benchmark.md)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchenxyzl%2Fmongoincupdate.fody","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchenxyzl%2Fmongoincupdate.fody","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchenxyzl%2Fmongoincupdate.fody/lists"}