{"id":26849258,"url":"https://github.com/nucs/nucs.essentials","last_synced_at":"2025-05-04T08:40:08.632Z","repository":{"id":60777000,"uuid":"544721452","full_name":"Nucs/Nucs.Essentials","owner":"Nucs","description":"Multithreading, Collections, Optimization, Span Text, Runtime Generation and other classes you might find essential.","archived":false,"fork":false,"pushed_at":"2024-05-11T15:04:57.000Z","size":485,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-04T01:58:37.060Z","etag":null,"topics":["bayesian-optimization","collection","collections","multhithreading","optimization","parallelism","struct","unmanaged"],"latest_commit_sha":null,"homepage":"","language":"C#","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Nucs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-03T05:26:25.000Z","updated_at":"2024-05-11T15:05:02.000Z","dependencies_parsed_at":"2024-05-11T16:24:59.585Z","dependency_job_id":"b34a491e-464e-415b-8282-20298231267d","html_url":"https://github.com/Nucs/Nucs.Essentials","commit_stats":{"total_commits":119,"total_committers":1,"mean_commits":119.0,"dds":0.0,"last_synced_commit":"e0df878b24478b97c214a42f1ead5042d7cdff79"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nucs%2FNucs.Essentials","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nucs%2FNucs.Essentials/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nucs%2FNucs.Essentials/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Nucs%2FNucs.Essentials/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Nucs","download_url":"https://codeload.github.com/Nucs/Nucs.Essentials/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252310933,"owners_count":21727512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian-optimization","collection","collections","multhithreading","optimization","parallelism","struct","unmanaged"],"created_at":"2025-03-30T21:32:54.212Z","updated_at":"2025-05-04T08:40:08.601Z","avatar_url":"https://github.com/Nucs.png","language":"C#","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cimg src=\"https://i.imgur.com/BOExs52.png\" width=\"25\" style=\"margin: 5px 0px 0px 10px\"/\u003e Nucs.Essentials\n[![Nuget downloads](https://img.shields.io/nuget/vpre/Nucs.Essentials.svg)](https://www.nuget.org/packages/Nucs.Essentials/)\n[![NuGet](https://img.shields.io/nuget/dt/Nucs.Essentials.svg)](https://github.com/Nucs/Nucs.Essentials)\n[![GitHub license](https://img.shields.io/github/license/mashape/apistatus.svg)](https://github.com/Nucs/Essentials/blob/master/LICENSE)\n\nIf you had a bunch of high performance classes, would you not place them in a nuget package?\u003cbr/\u003e\nThis library contains essential classes I use in production.\u003cbr/\u003e\nCloning and exploring this repository is the recommended way of learning how to use it.\n\n### Installation\nSupports `netcoreapp3.1` `net6.0` `net7.0`\n```sh\nPM\u003e Install-Package Nucs.Essentials\n```\nOverview\n---\nAll performance-oriented classes have a benchmark at [Nucs.Essentials.Benchmark](https://github.com/Nucs/Nucs.Essentials/tree/main/benchmark/Nucs.Essentials.Benchmarks) project and usually unit-tested.\u003cbr/\u003e\n\n### Text\n- [LineReader](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Streams/LineReader.cs) ([faster by 185% than string.Split](https://github.com/Nucs/Nucs.Essentials/blob/main/benchmark/Nucs.Essentials.Benchmarks/LineReaderBenchmark.cs) and [parses 25% faster than with string.Split](https://github.com/Nucs/Nucs.Essentials/blob/main/benchmark/Nucs.Essentials.Benchmarks/LineReaderParseBenchmark.cs))\n  allows splitting a string without creating a copy (using `Span\u003cchar\u003e`) with any kind of separator. Useful for csv parsing.\n- [RowReader](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Streams/RowReader.cs) ([faster by 215% than string.Split](https://github.com/Nucs/Nucs.Essentials/blob/main/benchmark/Nucs.Essentials.Benchmarks/RowReaderBenchmark.cs)) \n  similar to LineReader but specializes in row splitting without copy (using `Span\u003cchar\u003e`).\n- [StreamRowReader](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Streams/StreamRowReader.cs)\n    similar to RowReader but for streams with an automated buffer algorithm.\n- [ValueStringBuilder](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Text/ValueStringBuilder.cs) \n  allows building strings using pooled buffers, useful for high performance string building.\n- [ReverseLineReader](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Streams/StreamRowReader.cs)\n  for reading lines from the end of a file.\n\n### Collections\n- [RollingWindow\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/RollingWindow.cs)\n  a rolling window (list) of fixed size. When full, last one pops and new item is pushed to front, useful for statistics.\n- [StructList\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/Structs/StructList.cs) and [StructQueue\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/Structs/StructQueue.cs) \n  are struct port of `List\u003cT\u003e`/`Queue\u003cT\u003e` with additional functionalities such as exposing internal fields and deconstructors, essentially allowing a very versatile use of them.\n  Versioning to protect against multithreaded access has been removed.\n- Reusable queues for wrapping [List\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/Structs/ReusableListQueue.cs) / [Array](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/Structs/ReusableArrayQueue.cs) / [ReadOnlySpan\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/Structs/ReusableSpanQueue.cs) \n  allowing reuse/resetting the queue without needing to create a new instance. Also exposes functionalities such as Peak and iteration.\n\n### Multithreading / Collections\n- [Async](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/AsyncSingleProducerSingleConsumerQueue.cs) / [SingleProducerSingleConsumerQueue\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/SingleProducerSingleConsumerQueue.cs) \n  a high performance lockless queue for single producer and single consumer with awaitable signal for available read [faster than System.Threading.Channels by 81%](https://github.com/Nucs/Nucs.Essentials/blob/main/benchmark/Nucs.Essentials.Benchmarks/AsyncSingleProducerSingleConsumerQueue_EnqueueDequeue_Benchmark.cs).\n- [Async](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/AsyncManyProducerManyConsumerStack.cs) / [ManyProducerManyConsumerStack\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/ManyProducerManyConsumerStack.cs)\n  a high performance lockless stack using linked-list for many producers and many consumers with awaitable signal for available read.\n- [AsyncRoundRobinProducerConsumer\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/AsyncRoundRobinProducerConsumer.cs)\n  a lockless round-robin channel that accepts data from multiple producers and distributes it to multiple `AsyncSingleProducerSingleConsumerQueue` consumers. This pattern allows feeding `\u003cT\u003e` to multiple consumers without locking.\n- [AsyncCountdownEvent](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Threading/AsyncCountdownEvent.cs) \n  a lockless countdown event that allows awaiting for a specific number of signals. awaiting completes once 0 is reached. Counter can be incremented and decremented. Serves like a `SemaphoreSlim` that awaits for reaching 0 signals remaining.\n- [ConcurrentPriorityQueue\\\u003cTKey, TValue\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/ConcurrentPriorityQueue.cs)\n  lock based priority queue based on generic key ordering priority by a `IComparable\\\u003cTKey\\\u003e`.\n- [ConcurrentHashSet\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/ConcurrentHashSet.cs)\n    bucket-based locking (multiple locks, depending on hash of the item, better than single-lock) with Dictionary-like buckets hashset for concurrent access.\n- [ObservableConcurrentList](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Collections/ObservableConcurrentList.cs)\n    a thread-safe observable list that notifies changes using `INotifyCollectionChanged`. Useful for concurrent WPF binding. Allows transactions using `IDisposable BlockReentrancy()` that on dispose will notify changes.\n\n### Reflection / Generators / Expressions\nAll expression related classes have an overload for `Expression` and a `Delegate`.\n- [DictionaryToSwitchCaseGenerator](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Reflection/DictionaryToSwitchCaseGenerator.cs)\n  creates a switch-case expression from a dictionary of `TKey` to `TValue` and a default value case. Essentially inlines a dictionary into a switch-case as a `Func\u003cTKey, TValue\u003e`.\n- [PreloadedPropertyGetter](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Reflection/PreloadedPropertyGetter.cs)\n  generates a getter for all properties of a type and caches it for future use. Useful for reflection-heavy code.\n- [StructToString](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Reflection/StructToString.cs)\n  generates a `ToString` method for a struct that returns a string. Used to avoid a mistake of using `object.ToString` that forces a cast from struct to object.\n- [ToDictionaryGenerator](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Reflection/ToDictionaryGenerator.cs)\n  generates a `ToDictionary` method for a target type `\u003cT\u003e` that returns a `Dictionary\u003cstring, object\u003e` of all properties. Supports boxing of struct/primitive values via `PooledStrongBox\u003cT\u003e`. Useful for destructing an object into a dictionary.\n- [DefaultValue\\\u003cT\\\u003e](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Essentials/Reflection/DefaultValue.cs)\n  provides a method to create a default value of a `\u003cT\u003e`. As-well as a cached boxed value and `T` value.\n\n\n# \u003cimg src=\"https://i.imgur.com/BOExs52.png\" width=\"25\" style=\"margin: 5px 0px 0px 10px\"/\u003e Nucs.Optimization\n[![Nuget downloads](https://img.shields.io/nuget/vpre/Nucs.Optimization.svg)](https://www.nuget.org/packages/Nucs.Essentials/)\n[![NuGet](https://img.shields.io/nuget/dt/Nucs.Optimization.svg)](https://github.com/Nucs/Nucs.Essentials)\n[![GitHub license](https://img.shields.io/github/license/mashape/apistatus.svg)](https://github.com/Nucs/Essentials/blob/master/LICENSE)\n\n\nA .NET binding using [pythonnet](https://github.com/pythonnet/pythonnet) for [skopt (scikit-optimize)](https://scikit-optimize.github.io/) - an optimization library with support to dynamic search spaces through generic binding.\u003cbr/\u003e\n\nAvailable Algorithms:\n- [x] [Random Search](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Optimization/PyRandomOptimization.cs)\n- [x] [Bayesian Optimization](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Optimization/PyBayesianOptimization.cs)\n- [x] [Random Forest Optimization](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Optimization/PyForestOptimization.cs)\n- [x] [Gradient Boosting Regression Trees (Gbrt)](https://github.com/Nucs/Nucs.Essentials/blob/main/src/Nucs.Optimization/PyGbrtOptimization.cs)\n\n*Source code can be, [found here](https://github.com/Nucs/Nucs.Essentials/tree/main/src/Nucs.Optimization)*\n\n\n### Installation\n\n* Python 3.8+\n    ```sh\n    numpy\u003e=1.23.5\n    pythonnet\u003e=3.0.1\n    scikit-learn\u003e=1.2.0\n    scikit-optimize\u003e=0.9.0\n    scipy\u003e=1.9.3\n  \n    \u003e pip install numpy pythonnet scikit-learn scikit-optimize scipy\n    ```\n* .NET 7.0\n    ```sh\n    PM\u003e Install-Package Nucs.Optimization\n    ```\n\n### Getting Started\n\n\nDeclare a parameters class/record for the optimization search space.\u003c/br\u003e\nAnnotate it with IntegerSpace / RealSpace / CategoricalSpace attributes. \u003c/br\u003e\nNon-annotated parameters will be implicitly included by default.\n\n```C#\n\n[Parameters(Inclusion = ParametersInclusion.ImplicitAndExplicit)] //include all annotated and non-annotated\npublic record Parameters {\n    [IntegerSpace\u003cint\u003e(1, int.MaxValue, Prior = Prior.LogUniform, Base = 2, Transform = NumericalTransform.Normalize)]\n    public int Seed; //range of 0 to int.MaxValue (including)\n\n    [RealSpace\u003cdouble\u003e(0, Math.PI)]\n    public double FloatSeed; //range of 0 to int.MaxValue (including)\n\n    [CategoricalSpace\u003cfloat\u003e(1f, 2f, 3f)]\n    public float NumericalCategories { get; set; } //one of 1f, 2f, 3f\n\n    [CategoricalSpace\u003cdouble\u003e(1d, 10d, 100d, 1000d)]\n    public double LogNumericalCategories { get; set; } //one of 1d, 10d, 100d, 1000d\n\n    [CategoricalSpace\u003cstring\u003e(\"A\", \"B\", \"C\", Transform = CategoricalTransform.Identity)]\n    public string Categories; //one of \"A\", \"B\", \"C\"\n\n    [CategoricalSpace\u003cbool\u003e] //optional, will be included implicitly\n    public bool UseMethod; //true or false\n\n    [CategoricalSpace\u003cSomeEnum\u003e(SomeEnum.A, SomeEnum.B, SomeEnum.C)]\n    public SomeEnum AnEnum; //one of the enum values (\"A\", \"B\", \"C\")\n\n    /// string will be parsed to SomeEnum. Prior provides the priority of each possible value. 'B' will have 80% priority of being selected.\n    [CategoricalSpace\u003cSomeEnum\u003e(\"A\", \"B\", Prior = new double[] {0.2, 0.8})] \n    public SomeEnum AnEnumWithValues; //one of the enum values (\"A\", \"B\")\n\n    public SomeEnum AllValuesOfEnum; //one of any of the values of the enum\n        \n    [IgnoreDataMember]\n    public bool Ignored; //will be ignored entirely\n}\n\npublic enum SomeEnum { A, B, C }\n\n```\n\n```C#\n//setup python runtime\nRuntime.PythonDLL = Environment.ExpandEnvironmentVariables(\"%APPDATA%\\\\..\\\\Local\\\\Programs\\\\Python\\\\Python38\\\\python38.dll\");\nPythonEngine.Initialize();\nPythonEngine.BeginAllowThreads();\nusing var py = Py.GIL(); //no GIL is being taken inside. has to be taken outside.\n\n//declare a function to optimize\n[Maximize] //or [Minimize]\ndouble ScoreFunction(Parameters parameters) {\n    return (parameters.Seed * parameters.NumericalCategories * (parameters.UseMethod ? 1 : -1) * Math.Sin(0.05+parameters.FloatSeed)) / 1000000;\n}\n\n//construct an optimizer\nvar opt = new PyBayesianOptimization\u003cParameters\u003e(ScoreFunction);\nvar opt2 = new PyForestOptimization\u003cParameters\u003e(ScoreFunction);\nvar opt3 = new PyRandomOptimization\u003cParameters\u003e(ScoreFunction);\nvar opt4 = new PyGbrtOptimization\u003cParameters\u003e(ScoreFunction);\n\n//(optional) prepare callbacks\nvar callbacks = new PyOptCallback[] { new IterationCallback\u003cParameters\u003e(maximize: true, (iteration, parameters, score) =\u003e {\n    Console.WriteLine($\"[{iteration}] Score: {score}, Parameters: {parameters}\");\n})};\n\n//run optimizer of choice (Search, SearchTop, SearchAll)\ndouble Score;\nParameters Parameters;\n(Score, Parameters) = opt.Search(n_calls: 100, n_random_starts: 10, verbose: false, callbacks: callbacks);\n(Score, Parameters) = opt2.Search(n_calls: 100, n_random_starts: 10, verbose: false, callbacks: callbacks);\n(Score, Parameters) = opt3.Search(n_calls: 100, verbose: false, callbacks: callbacks);\n(Score, Parameters) = opt4.Search(n_calls: 100, n_random_starts: 10, verbose: false, callbacks: callbacks);\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnucs%2Fnucs.essentials","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnucs%2Fnucs.essentials","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnucs%2Fnucs.essentials/lists"}