{"id":13936353,"url":"https://github.com/alejandrodumas/kodiak","last_synced_at":"2025-07-19T21:32:26.943Z","repository":{"id":24016230,"uuid":"100397286","full_name":"alejandrodumas/kodiak","owner":"alejandrodumas","description":"Enhance your feature engineering workflow with Kodiak","archived":false,"fork":false,"pushed_at":"2023-08-02T01:28:18.000Z","size":42,"stargazers_count":19,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-06-18T12:57:14.730Z","etag":null,"topics":["data-analysis","pandas"],"latest_commit_sha":null,"homepage":"http://kodiak.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alejandrodumas.png","metadata":{"files":{"readme":"README.rst","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-15T16:31:37.000Z","updated_at":"2024-11-28T16:33:30.000Z","dependencies_parsed_at":"2024-04-27T23:53:34.195Z","dependency_job_id":"1fdc0156-2acc-4b61-80bc-59997a7def87","html_url":"https://github.com/alejandrodumas/kodiak","commit_stats":{"total_commits":8,"total_committers":2,"mean_commits":4.0,"dds":0.125,"last_synced_commit":"963278c29ac9575f306cdbd1722ef9aeedb82152"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/alejandrodumas/kodiak","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrodumas%2Fkodiak","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrodumas%2Fkodiak/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrodumas%2Fkodiak/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrodumas%2Fkodiak/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alejandrodumas","download_url":"https://codeload.github.com/alejandrodumas/kodiak/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrodumas%2Fkodiak/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265896297,"owners_count":23845457,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","pandas"],"created_at":"2024-08-07T23:02:35.462Z","updated_at":"2025-07-19T21:32:21.934Z","avatar_url":"https://github.com/alejandrodumas.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Kodiak\n======\n\n.. image:: https://img.shields.io/github/license/mashape/apistatus.svg\n\n.. image:: https://readthedocs.org/projects/kodiak/badge/?version=latest\n    :target: http://kodiak.readthedocs.io/en/latest/?badge=latest\n\n\nOverview\n--------\n\n**Kodiak** enhances your feature engineering workflow extracting common\npatterns so you can create more features faster.\n\nEx: You have the ``writers`` dataframe, where ``born`` is a datetime\n\n+-----------------------+--------------+\n| name                  | born         |\n+=======================+==============+\n| Miguel de Cervantes   | 09-29-1547   |\n+-----------------------+--------------+\n| William Shakespeare   | 04-23-1617   |\n+-----------------------+--------------+\n\nand you want to extract from the ``born`` column: ``day``, ``month`` and\n``year`` and create 3 new columns\n\n+-----------------------+--------------+---------------+-------------+--------------+\n| name                  | born         | born\\_month   | born\\_day   | born\\_year   |\n+=======================+==============+===============+=============+==============+\n| Miguel de Cervantes   | 09-29-1547   | 9             | 29          | 1547         |\n+-----------------------+--------------+---------------+-------------+--------------+\n| William Shakespeare   | 04-23-1617   | 4             | 23          | 1617         |\n+-----------------------+--------------+---------------+-------------+--------------+\n\nThe simplest thing you could do in Pandas is:\n\n.. code:: python\n\n    writers[\"born_month\"] = writers.born.map(lambda x: x.month)\n    writers[\"born_day\"]   = writers.born.map(lambda x: x.day)\n    writers[\"born_year\"]  = writers.born.map(lambda x: x.year)\n\nWith Kodiak you could get the same result in one line:\n\n.. code:: python\n\n    writers.gencol(\"born_{month,day,year}\", \"born\", lambda x, y: getattr(x, y))\n    # or more succinctly\n    writers.gencol(\"born_{.month,.day,.year}\", \"born\")\n\nBut, how does it work? Kodiak uses ``\"born_{month,day,year}\"`` as a\ntemplate for the columns: ``born_month``, ``born_day`` and ``born_year``\nand passes ``month``,\\ ``day`` and ``year`` as arguments to a provided\nfunction that also has the current 'born' as an an argument, so you're\nbasically doing:\n\n.. code:: python\n\n    for y in ['month', 'day', 'year']:\n        writers[\"born_{}\".format(y)] = writers.born.map(lambda x: getattr(x, y))\n\nKodiak does a lot of other things to boost your workflow, for that, see\nthe *Basic Usage* and *Advanced Usage* sections\n\nInstallation\n------------\n\nTo install Kodiak, ``cd`` to the Kodiak folder and run the install\ncommand:\n\n.. code:: sh\n\n    sudo python setup.py install\n\nYou can also install Kodiak from PyPI:\n\n.. code:: sh\n\n    sudo pip install kodiak\n\nBasic Usage\n-----------\n\n**Kodiak** main object is ``KodiakDataFrame`` an extension of\n``pandas.Dataframe`` that provides the instance method ``colgen`` to\ncreate one or more columns. You create ``KodiakDataFrames`` the same way\nyou create a ``pandas.DataFrame``\n\n``colgen`` signature is:\n``colgen(newcols, col, colbuilder=None, enum=False, config=None)``\n\nnewcols\n  has a double function, it works as a specification of the columns you\n  want to create, and it also contains the values passed to ``colbuilder``\n\n  .. code:: python\n\n      # If you want to create the columns `first_name`, `last_name`\n      # and pass `first` and `last` as arguments to `colbuilder` you write\n      \u003e\u003e\u003e newcols = \"{first,last}_name\"\n\n      # More complex patterns allowed\n      \u003e\u003e\u003e groups = \"col_{a,b}_{c,d}\"\n\n      # Will create the columns: `col_a_c`, `col_b_d`\n      # The way `a,b` and `c,d` is combined can be configured\n\ncol\n  is the `KodiakDataframe` column from where you'll extract information\n  to create your new column/s\n\ncolbuilder\n  is a function or lambda used to extract info from ``col``\n  and create the columns specified in ``newcols`` with the\n  corresponding ``col`` instance and the ``newcols`` values.\n  The signature of `colbuilder` is `colbuilder(x, y)` or\n  `colbuilder(i, x, y)` `x` is an instance of the column passed\n  in `col` and `y` is an argument extracted from `newcols`. The\n  extra argument `i` is an index of the arguments.\n\nconfig\n  tweak Kodiak inner workings with your own config, see the\n  dedicated section for more info\n\nAdvanced Usage\n--------------\n\nIn this section we're going to describe the main components and concepts\nthat are essential to Kodiak\n\nTemplating\n~~~~~~~~~~\n\nThe template language is minimal but has some extensions to help you:\n\nRanges\n^^^^^^\n\nThe range notation is ``start:end:step``. Reverse ranges are permitted\nsetting ``end`` bigger than ``start``. ``step`` default value is ``1``, and\n``start`` is ``0``, finally if ``end`` is absent, it'll be setted to ``0`` and\nyou'll have a reversed range. Ranges are inclusive.\n\n.. code:: python\n\n    simple_range = \"col_{1:3}\" # -\u003e col_1, col_2, col_3\n    step_range = \"col_{:3:2}\" # -\u003e col_0, col_3\n    inverse_range = \"col_{3:1}\" # -\u003e col_3,col_2,col_1\n    no_end = \"2:\" # -\u003e col_2,col_1,col_0\n\nKey-Value\n^^^^^^^^^\n\nIf you want the column name and argument passed to the ``colbuilder`` to\ndiffer you can use key-values.\n\n.. code:: python\n\n    dataframe.gencol(\"{short=very_long_name}_col\", \"col\", alambda)\n    # In this case the column name will be ``short_col`` but you'll pass\n    # ``very_long_name`` to ``alambda``\n\n    # key-value notation can be extended to more arguments:\n    dataframe.gencol(\"{k1=v1,k2=v2,k3=v3}_col\", \"col\", alambda)\n\n.. WARNING::\n  values are always interpreted as *strings* so in:\n  ``col_{k=1:5}`` the value ``1:5`` is interpreted as ``\"1:5\"`` and not as\n  a range, the same for ``col_{k=[1,2,3]}`` and any other object, also if\n  you pass a number it will also be interpreted as string so you will need\n  to convert it if you intend to use it as an ``int``.\n\nTransforms\n~~~~~~~~~~\n\nUnder the hood when you pass ``newcols`` to ``gencol``, Kodiak builds an\n``OrderedDict`` where it's keys are column names and it's values are\ntuples of ``Match`` objects -even if it's just one Match it's wrapped\ninside a tuple-\n\n.. code:: python\n\n    newcol = \"{first,last}_name\"\n    # will build\n    args_dict = {'first_name': (Match('first'),), 'last_name': (Match('last'),)}\n\n``Transforms`` are a way to pre-process the values and change them\nenriching the ``Match`` object with a payload as you will see in the\n``Default colbuilder`` section.\n\nSo, if the values are ``Match`` objects, how is that when you write your\n``colbuilder`` you deal with ``strings``? Kodiak understands that if the\n``Match`` object doesn't have a payload it's better to pass strings\narguments to ``colbuilder``, this behaviour can be controlled.\n\nWhat's the use of ``Match`` objects and their ``payload``? What're some\nexamples of ``Transforms``? The next section will answer this questions\n\nDefault colbuilder\n~~~~~~~~~~~~~~~~~~\n\nAs you can see in the ``colgen`` signature, ``colbuilder`` default\nargument is ``None``, in special cases Kodiak can infer the\n``colbuilder`` method, let's revisit the opening example.\n\n.. code:: python\n\n    writers.gencol(\"born_{.month,.day,.year}\", \"born\")\n\nThe ``colbuilder`` in this case is inferred from the hint you gave\nKodiak in the template: ``.month``, prefixing ``month`` with a ``.``\nindicates that you're referring to an attribute of ``born``, so\ninternally Kodiak builds a ``colbuider`` that extracts the ``month``\nfrom a ``born`` instance. Another way of omitting the ``colbuilder`` is\nwhen you have an instance method:\n\n.. code:: python\n\n    # Notice the `!` after weekday\n    writers.gencol(\"born_{weekday!}\", \"born\")\n\n.. WARNING::\n  This hint only works for methods with no arguments, passing\n  a method with one or more arguments will raise an error\n\nHow is that Kodiak infers the ``colbuilder``? When the ``newcols`` are\nprocessed they go through a pipeline of ``Transforms``, one of them:\n``PropertyTransform`` detects that ``.month`` refers to an attribute and\nenriches de ``Match`` object hinting in the payload the corresponding\n``colbuilder``, that's why you don't need to pass the ``colbuilder``\nargument. But what happens if you give a ``colbuilder``? In this case,\nas the ``Match`` object has a ``payload`` instead of working with plain\nstrings you will work with tuples of ``Match`` objects\n\n.. Note::\n  Kodiak will raise an exception when it can't figure out a\n  default colbuilder\n\nEnumerations\n~~~~~~~~~~~~\n\nSometimes you care about the position of the arguments not the exact\nvalue, in that case you can use the ``enum`` param or the implicit\n``enum`` with a function or lambda of arity 3, the first argument will\nbe the index starting at 0.\n\n.. code:: python\n\n    writers.gencol(\"{first=0,last=1}_name\", \"name\", lambda x,y: x.split(\" \")[int(y)])\n\n    # Another way with enum=True\n    writers.gencol(\"{first,last}_name\", \"name\", lambda i,x,y: x.split(\" \")[i], enum=True)\n\n    # Without enum=True but with a colbuilder with arity 3\n    writers.gencol(\"{first,last}_name\", \"name\", lambda i,x,y: x.split(\" \")[i])\n\nConfiguration\n-------------\n\nAlmost everything is configurable in Kodiak, you could have a per-method\nconfiguration or system-wide config.\n\nThe ``Config`` object has the following customizable params:\n\nparser\n  Kodiak by default uses the ``ArgsParser`` class to parse ``newcols``\n\nmatch\\_transform\n  data passed to the ``colbuilder`` could be\n  transformed first, by default we use the ``default_transform`` pipeline,\n  you could replace it with an array of ``Transforms`` objects.\n\nnew\\_col\\_combiner\n  in the newcols template if you have\n  ``\"col_{a,b}_{c,d}\"``, this results in the columns: ``\"col_a_c\"`` and\n  ``\"col_b,d\"``, how the different groups ``['a','b']`` and ``['c', 'd']``\n  are combined is controlled with this param, currently we use the ``zip``\n  function, and you could replace it with a function with similar\n  signature.\n\nunpack\n  Boolean Default True, when ``newcols`` is simple, ``foo_{a,b}``\n  instead of ``foo_{.a,b!}`` instead of passing to ``colbuilder``\n  tuples of ``Match`` objects we just pass individual items,\n  ``a``, ``b``, so it's easier to build a ``colbuilder`` without\n  having to unwrap the ``Match`` tuples\n\ncol\\_pair\\_combiner\n  Once you have the arguments that you're going to\n  pass to the ``colbuilder`` you can combine them in different ways, currently\n  we use ``product`` from itertools, ie: the cartesian product between an\n  element, ex: ``event``, and the other n-columns, creating the following\n  tuples:\n\n  .. code:: python\n\n      [('event', 'day') , ('event', 'month'), ('event', 'year')]\n\nYou can replace this method with another with the same signature as ``product``\n\nConfig can be accessed, modified and restored with:\n\n.. code:: python\n\n    \u003e\u003e import config\n    \u003e\u003e from config import cfg\n    \u003e\u003e config.options\n\n    # Global change on config\n\n    \u003e\u003e config.options[\"unpack\"] = False\n    \u003e\u003e config.options[\"col_pair_combiner\"] = zip\n\n    # Restoring one or more fields of the configuration\n    \u003e\u003e config.restore_default_config(\"col_pair_combiner\")\n\n    # Restoring all the options\n    \u003e\u003e config.restore_default_config()\n\n    # With `base_config` or it's alias `cfg` you can create modified versions\n    # of the default config\n\n    \u003e\u003e dataframe.gencol(\"col_{a!,b!}\",\"col\", func, config=cfg(unpack=False))\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falejandrodumas%2Fkodiak","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falejandrodumas%2Fkodiak","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falejandrodumas%2Fkodiak/lists"}