{"id":19557049,"url":"https://github.com/cereja-project/cereja","last_synced_at":"2025-04-06T22:06:38.641Z","repository":{"id":37012474,"uuid":"208130874","full_name":"cereja-project/cereja","owner":"cereja-project","description":"Cereja is a bundle of useful functions we don't want to rewrite and .. just pure fun!","archived":false,"fork":false,"pushed_at":"2024-10-21T22:43:46.000Z","size":664,"stargazers_count":27,"open_issues_count":3,"forks_count":12,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-22T19:04:27.913Z","etag":null,"topics":["array-manipulations","colab","console","data-tools","datapreprocessing","file-converter","freq","freqitems","hacktoberfest","hacktoberfest2024","progress-bar","progress-view","python","python-library","python3","tfidf","tokenizer","utilities"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cereja-project.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-12T19:31:14.000Z","updated_at":"2024-10-21T22:43:51.000Z","dependencies_parsed_at":"2023-02-14T14:31:27.235Z","dependency_job_id":"ae9a72f8-f111-4fcf-982d-68813f0769d9","html_url":"https://github.com/cereja-project/cereja","commit_stats":null,"previous_names":[],"tags_count":114,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cereja-project%2Fcereja","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cereja-project%2Fcereja/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cereja-project%2Fcereja/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cereja-project%2Fcereja/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cereja-project","download_url":"https://codeload.github.com/cereja-project/cereja/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247557767,"owners_count":20958047,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array-manipulations","colab","console","data-tools","datapreprocessing","file-converter","freq","freqitems","hacktoberfest","hacktoberfest2024","progress-bar","progress-view","python","python-library","python3","tfidf","tokenizer","utilities"],"created_at":"2024-11-11T04:40:03.670Z","updated_at":"2025-04-06T22:06:38.621Z","avatar_url":"https://github.com/cereja-project.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cereja 🍒\n\n![Python package](https://github.com/jlsneto/cereja/workflows/Python%20package/badge.svg)\n[![PyPI version](https://badge.fury.io/py/cereja.svg)](https://badge.fury.io/py/cereja)\n[![Downloads](https://pepy.tech/badge/cereja)](https://pepy.tech/project/cereja)\n[![MIT LICENSE](https://img.shields.io/pypi/l/pyzipcode-cli.svg)](LICENSE)\n[![Issues](https://camo.githubusercontent.com/926d8ca67df15de5bd1abac234c0603d94f66c00/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f636f6e747269627574696f6e732d77656c636f6d652d627269676874677265656e2e7376673f7374796c653d666c6174)](https://github.com/jlsneto/cereja/issues/new/choose)\n[![Get start on Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/jlsneto/cereja/blob/master/docs/cereja_example.ipynb)\n\n\u003cdiv align=\"center\"\u003e\n \u003cimg src=\"https://i.ibb.co/Fw8SSfd/cereja-logo.png\" height=\"300\" width=\"300\" alt=\"CEREJA\"\u003e\n\u003c/div\u003e\n\n*Cereja was written only with the Standard Python Library, and it was a great way to improve knowledge in the Language\nalso to avoid the rewriting of code.*\n\n## Getting Started DEV\n\nDon't be shy \\0/ ... Clone the repository and submit a function or module you made or use some function you liked.\n\nSee [CONTRIBUTING](CONTRIBUTING.md) 💻\n\n## Setup\n\n* [Python 3.6+](https://www.python.org/downloads/ \"Download python\")\n* [Pip3](https://pip.pypa.io \"Download Pip\")\n\n## Install\n\n```\npip install --user cereja\n```\n\nor for all users\n\n```\npip install cereja\n```\n\n## Cereja Example usage\n\nSee some of the Cereja tools\n\nTo access the *Cereja's* tools you need to import it `import cereja as cj`.\n\n### 📝 [FileIO](docs/file.md)\n\n#### Create new files\n\n```python\nimport cereja as cj\n\nfile_json = cj.FileIO.create('./json_new_file.json', data={'k': 'v', 'k2': 'v2'})\n\nfile_txt = cj.FileIO.create('./txt_new_file.txt', ['line1', 'line2', 'line3'])\n\nfile_json.save()\nfile_txt.save()\n\nprint(file_json.exists)\n# True\nprint(file_txt.exists)\n# True\n\n\n# see what you can do .txt file\nprint(cj.can_do(file_txt))\n\n# see what you can do .json file\nprint(cj.can_do(file_json))\n```\n\n#### Load and edit files\n\n```python\nimport cereja as cj\n\nfile_json = cj.FileIO.load('./json_new_file.json')\n\nprint(file_json.data)\n# {'k': 'v', 'k2': 'v2'}\n\nfile_json.add(key='new_key', value='value')\nprint(file_json.data)\n# {'k': 'v', 'k2': 'v2', 'new_key': 'value'}\n\nfile_txt = cj.FileIO.load('./txt_new_file.txt')\n\nprint(file_txt.data)\n# ['line1', 'line2', 'line3']\n\nfile_txt.add('line4')\nprint(file_txt.data)\n# ['line1', 'line2', 'line3', 'line4']\n\nfile_txt.save(exist_ok=True)  # Override\nfile_json.save(exist_ok=True)  # Override\n```\n\n### 📍 Path\n\n```python\nimport cereja as cj\n\nfile_path = cj.Path('/my/path/file.ext')\nprint(cj.can_do(file_path))\n# ['change_current_dir', 'cp', 'created_at', 'exists', 'get_current_dir', 'is_dir', 'is_file', 'is_hidden', 'is_link', 'join', 'last_access', 'list_dir', 'list_files', 'mv', 'name', 'parent', 'parent_name', 'parts', 'path', 'rm', 'root', 'rsplit', 'sep', 'split', 'stem', 'suffix', 'updated_at', 'uri']\n```\n\n### 🆗 HTTP Requests\n\n```python\nimport cereja as cj\n\n# Change url, headers and data values.\nurl = 'localhost:8000/example'\nheaders = {'Authorization': 'TOKEN'} # optional\ndata = {'q': 'test'} # optional\n\nresponse = cj.request.post(url, data=data, headers=headers)\n\nif response.code == 200:\n    data = response.data\n    # have a fun!\n```\n\n### ⏳ [Progress](docs/display.md)\n\n```python\nimport cereja as cj\nimport time\n\nmy_iterable = ['Cereja', 'is', 'very', 'easy']\n\nfor i in cj.Progress.prog(my_iterable):\n    print(f\"current: {i}\")\n    time.sleep(2)\n\n# Output on terminal ...\n\n# 🍒 Sys[out] » current: Cereja \n# 🍒 Sys[out] » current: is \n# 🍒 Cereja Progress » [▰▰▰▰▰▰▰▰▰▰▰▰▰▰▰▱▱▱▱▱▱▱▱▱▱▱▱▱▱] - 50.00% - 🕢 00:00:02 estimated\n\n\n```\n\n### 🧠 [Data Preparation](docs/ml.md)\n\n📊 **Freq**\n\n```python\nimport cereja as cj\n\nfreq = cj.Freq([1, 2, 3, 3, 10, 10, 4, 4, 4, 4])\n# Output -\u003e Freq({1: 1, 2: 1, 3: 2, 10: 2, 4: 4})\n\nfreq.most_common(2)\n# Output -\u003e {4: 4, 3: 2}\n\nfreq.least_freq(2)\n# Output -\u003e {2: 1, 1: 1}\n\nfreq.probability\n# Output -\u003e OrderedDict([(4, 0.4), (3, 0.2), (10, 0.2), (1, 0.1), (2, 0.1)])\n\nfreq.sample(min_freq=1, max_freq=2)\n# Output -\u003e {3: 2, 10: 2, 1: 1, 2: 1}\n\n# Save json file.\nfreq.to_json('./freq.json')\n```\n\n🧹 **Text Preprocess**\n\n```python\nimport cereja as cj\n\ntext = \"Oi tudo bem?? meu nome é joab!\"\n\ntext = cj.preprocess.remove_extra_chars(text)\nprint(text)\n# Output -\u003e 'Oi tudo bem? meu nome é joab!'\n\ntext = cj.preprocess.separate(text, sep=['?', '!'])\n# Output -\u003e 'Oi tudo bem ? meu nome é joab !'\n\ntext = cj.preprocess.accent_remove(text)\n# Output -\u003e 'Oi tudo bem ? meu nome e joab !'\n\n# and more ..\n\n# You can use class Preprocessor ...\npreprocessor = cj.Preprocessor(stop_words=(),\n                               punctuation='!?,.', to_lower=True, is_remove_punctuation=False,\n                               is_remove_stop_words=False,\n                               is_remove_accent=True)\n\nprint(preprocessor.preprocess(text))\n# Output -\u003e 'oi tudo bem ? meu nome e joab !'\n\nprint(preprocessor.preprocess(text, is_destructive=True))\n# Output -\u003e 'oi tudo bem meu nome e joab'\n\n```\n\n🔣 **Tokenizer**\n\n```python\nimport cereja as cj\n\ntext = ['oi tudo bem meu nome é joab']\n\ntokenizer = cj.Tokenizer(text, use_unk=True)\n\n# tokens 0 to 9 is UNK\n# hash_ used to replace UNK\ntoken_sequence, hash_ = tokenizer.encode('meu nome é Neymar Júnior')\n# Output -\u003e [([10, 12, 11, 0, 1], 'eeb755960ce70c')]\n\ndecoded_sequence = tokenizer.decode(token_sequence, hash_=hash_)\n# Output -\u003e 'meu nome é Neymar Júnior'\n\n```\n\n⏸ **Corpus**\n\nGreat training and test separator.\n\n```python\nimport cereja as cj\n\nX = ['how are you?', 'my name is Joab', 'I like coffee', 'how are you joab?', 'how', 'we are the world']\nY = ['como você está?', 'meu nome é Joab', 'Eu gosto de café', 'Como você está joab?', 'como', 'Nós somos o mundo']\n\ncorpus = cj.Corpus(source_data=X, target_data=Y, source_name='en', target_name='pt')\nprint(corpus)  # Corpus(examples: 6 - source_vocab_size: 13 - target_vocab_size:15)\nprint(corpus.source)  # LanguageData(examples: 6 - vocab_size: 13)\nprint(corpus.target)  # LanguageData(examples: 6 - vocab_size: 15)\n\ncorpus.source.phrases_freq\n# Counter({'how are you': 1, 'my name is joab': 1, 'i like coffee': 1, 'how are you joab': 1, 'how': 1, 'we are the world': 1})\n\ncorpus.source.word_freq\n# Counter({'how': 3, 'are': 3, 'you': 2, 'joab': 2, 'my': 1, 'name': 1, 'is': 1, 'i': 1, 'like': 1, 'coffee': 1, 'we': 1, 'the': 1, 'world': 1})\n\ncorpus.target.phrases_freq\n# Counter({'como você está': 1, 'meu nome é joab': 1, 'eu gosto de café': 1, 'como você está joab': 1, 'como': 1, 'nós somos o mundo': 1})\n\ncorpus.target.words_freq\n# Counter({'como': 3, 'você': 2, 'está': 2, 'joab': 2, 'meu': 1, 'nome': 1, 'é': 1, 'eu': 1, 'gosto': 1, 'de': 1, 'café': 1, 'nós': 1, 'somos': 1, 'o': 1, 'mundo': 1})\n\n# split_data function guarantees test data without data identical to training\n# and only with vocabulary that exists in training\ntrain, test = corpus.split_data()  # default percent of training is 80%\n```\n\n### 🔢 Array\n\n```python\nimport cereja as cj\n\ncj.array.is_empty(data)  # False\ncj.array.get_shape(data)  # (2, 3)\n\ndata = cj.array.flatten(data)  # [1, 2, 3, 3, 3, 3]\ncj.array.prod(data)  # 162\ncj.array.sub(data)  # -13\ncj.array.div(data)  # 0.006172839506172839\n\ncj.array.rand_n(0.0, 2.0, n=3)  # [0.3001196087729699, 0.639679494102923, 1.060200897124107]\ncj.array.rand_n(1, 10)  # 5.086403830031244\ncj.array.array_randn((3, 3,\n                      3))  # [[[0.015077210355770374, 0.014298110484612511, 0.030410666810216064], [0.029319083335697604, 0.0072365209507707666, 0.010677361074992], [0.010576754075922935, 0.04146379877648334, 0.02188348813336284]], [[0.0451851551098092, 0.037074906805326824, 0.0032484586475421007], [0.025633380630695347, 0.010312669541918484, 0.0373624007621097], [0.047923908102496145, 0.0027939333359724224, 0.05976224377251878]], [[0.046869510719106486, 0.008325638358172866, 0.0038702998343255893], [0.06475268683502387, 0.0035638592537234623, 0.06551037943638163], [0.043317416824708604, 0.06579372884523939, 0.2477564291871006]]]\ncj.chunk(data=[1, 2, 3, 4], batch_size=3, fill_with=0)  # [[1, 2, 3], [4, 0, 0]]\ncj.array.remove_duplicate_items(['hi', 'hi', 'ih'])  # ['hi', 'ih'] \ncj.array.get_cols([['line1_col1', 'line1_col2'],\n                   ['line2_col1', 'line2_col2']])  # [['line1_col1', 'line2_col1'], ['line1_col2', 'line2_col2']]\ncj.array.dotproduct([1, 2], [1, 2])  # 5\n\na = cj.array.array_gen((3, 3), 1)  # [[1, 1, 1], [1, 1, 1], [1, 1, 1]]\nb = cj.array.array_gen((3, 3), 1)  # [[1, 1, 1], [1, 1, 1], [1, 1, 1]]\ncj.array.dot(a, b)  # [[3, 3, 3], [3, 3, 3], [3, 3, 3]]\ncj.mathtools.theta_angle((2, 2), (0, -2))  # 135.0\n\n```\n\n### 🧰 Utils\n\n```python\nimport cereja.utils.time\nimport cereja as cj\n\ndata = {\"key1\": 'value1', \"key2\": 'value2', \"key3\": 'value3', \"key4\": 'value4'}\n\ncj.utils.chunk(list(range(10)), batch_size=3)\n# [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]\ncj.utils.chunk(list(range(10)), batch_size=3, fill_with=0, is_random=True)\n# [[9, 7, 8], [0, 3, 2], [4, 1, 5], [6, 0, 0]]\n\n# Invert Dict\ncj.utils.invert_dict(data)\n# Output -\u003e {'value1': 'key1', 'value2': 'key2', 'value3': 'key3', 'value4': 'key4'}\n\n# Get sample of large data\ncj.utils.sample(data, k=2, is_random=True)\n# Output -\u003e {'key1': 'value1', 'key4': 'value4'}\n\ncj.utils.fill([1, 2, 3, 4], max_size=20, with_=0)\n# Output -\u003e [1, 2, 3, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]\n\ncj.utils.rescale_values([1, 2, 3, 4, 5, 6, 7, 8, 9, 10], granularity=4)\n# Output -\u003e [1, 3, 5, 7]\n\ncj.utils.import_string('cereja.file._io.FileIO')\n# Output -\u003e \u003cclass 'cereja.file._io.FileIO'\u003e\n\ncj.utils.list_methods(cj.Path)\n# Output -\u003e ['change_current_dir', 'cp', 'get_current_dir', 'join', 'list_dir', 'list_files', 'mv', 'rm', 'rsplit', 'split']\n\n\ncj.utils.string_to_literal('[1,2,3,4]')\n# Output -\u003e [1, 2, 3, 4]\n\ncereja.utils.time.time_format(3600)\n# Output -\u003e '01:00:00'\n\ncj.utils.truncate(\"Cereja is fun.\", k=3)\n# Output -\u003e 'Cer...'\n\ndata = [[1, 2, 3], [3, 3, 3]]\ncj.utils.is_iterable(data)  # True\ncj.utils.is_sequence(data)  # True\ncj.utils.is_numeric_sequence(data)  # True\n```\n\n[See Usage - Jupyter Notebook](./docs/cereja_example.ipynb)\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcereja-project%2Fcereja","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcereja-project%2Fcereja","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcereja-project%2Fcereja/lists"}