{"id":13411610,"url":"https://github.com/go-gota/gota","last_synced_at":"2025-05-13T21:12:16.028Z","repository":{"id":37548291,"uuid":"51212673","full_name":"go-gota/gota","owner":"go-gota","description":"Gota: DataFrames and data wrangling in Go (Golang)","archived":false,"fork":false,"pushed_at":"2023-08-09T06:59:36.000Z","size":538,"stargazers_count":3166,"open_issues_count":81,"forks_count":286,"subscribers_count":85,"default_branch":"master","last_synced_at":"2025-04-29T13:45:19.432Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/go-gota.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-02-06T17:23:25.000Z","updated_at":"2025-04-28T15:03:58.000Z","dependencies_parsed_at":"2023-01-30T21:31:14.273Z","dependency_job_id":"12be5c51-1fe5-407d-b381-33ce3e6d3e71","html_url":"https://github.com/go-gota/gota","commit_stats":{"total_commits":384,"total_committers":26,"mean_commits":14.76923076923077,"dds":0.1171875,"last_synced_commit":"f70540952827cfc8abfa1257391fd33284300b24"},"previous_names":["kniren/gota"],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-gota%2Fgota","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-gota%2Fgota/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-gota%2Fgota/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-gota%2Fgota/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/go-gota","download_url":"https://codeload.github.com/go-gota/gota/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254029008,"owners_count":22002284,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:14.971Z","updated_at":"2025-05-13T21:12:11.017Z","avatar_url":"https://github.com/go-gota.png","language":"Go","funding_links":[],"categories":["Go","Generators","DataFrames","Golang","Uncategorized"],"sub_categories":["Miscellaneous Data Structures and Algorithms","Tools","[Tools](#tools-1)","Vector Database","Numerical Libraries \u0026 Data Structures"],"readme":"Gota: DataFrames, Series and Data Wrangling for Go\n==================================================\n\nMeet us on Slack: Slack: [gophers.slack.com](https://gophers.slack.com) #go-gota ([invite](https://gophersinvite.herokuapp.com/))\n\nThis is an implementation of DataFrames, Series and data wrangling\nmethods for the Go programming language. The API is still in flux so\n*use at your own risk*.\n\nDataFrame\n---------\n\nThe term DataFrame typically refers to a tabular dataset that can be\nviewed as a two dimensional table. Often the columns of this dataset\nrefers to a list of features, while the rows represent a number of\nmeasurements. As the data on the real world is not perfect, DataFrame\nsupports non measurements or NaN elements.\n\nCommon examples of DataFrames can be found on Excel sheets, CSV files\nor SQL database tables, but this data can come on a variety of other\nformats, like a collection of JSON objects or XML files.\n\nThe utility of DataFrames resides on the ability to subset them, merge\nthem, summarize the data for individual features or apply functions to\nentire rows or columns, all while keeping column type integrity.\n\n### Usage\n#### Loading data\n\nDataFrames can be constructed passing Series to the dataframe.New constructor\nfunction:\n\n```go\ndf := dataframe.New(\n\tseries.New([]string{\"b\", \"a\"}, series.String, \"COL.1\"),\n\tseries.New([]int{1, 2}, series.Int, \"COL.2\"),\n\tseries.New([]float64{3.0, 4.0}, series.Float, \"COL.3\"),\n)\n```\n\nYou can also load the data directly from other formats. \nThe base loading function takes some records in the\nform `[][]string` and returns a new DataFrame from there:\n\n```go\ndf := dataframe.LoadRecords(\n    [][]string{\n        []string{\"A\", \"B\", \"C\", \"D\"},\n        []string{\"a\", \"4\", \"5.1\", \"true\"},\n        []string{\"k\", \"5\", \"7.0\", \"true\"},\n        []string{\"k\", \"4\", \"6.0\", \"true\"},\n        []string{\"a\", \"2\", \"7.1\", \"false\"},\n    },\n)\n```\n\nNow you can also create DataFrames by loading an slice of arbitrary structs:\n\n```go\ntype User struct {\n\tName     string\n\tAge      int\n\tAccuracy float64\n    ignored  bool // ignored since unexported\n}\nusers := []User{\n\t{\"Aram\", 17, 0.2, true},\n\t{\"Juan\", 18, 0.8, true},\n\t{\"Ana\", 22, 0.5, true},\n}\ndf := dataframe.LoadStructs(users)\n```\n\nBy default, the column types will be auto detected but this can be\nconfigured. For example, if we wish the default type to be `Float` but\ncolumns `A` and `D` are `String` and `Bool` respectively:\n\n```go\ndf := dataframe.LoadRecords(\n    [][]string{\n        []string{\"A\", \"B\", \"C\", \"D\"},\n        []string{\"a\", \"4\", \"5.1\", \"true\"},\n        []string{\"k\", \"5\", \"7.0\", \"true\"},\n        []string{\"k\", \"4\", \"6.0\", \"true\"},\n        []string{\"a\", \"2\", \"7.1\", \"false\"},\n    },\n    dataframe.DetectTypes(false),\n    dataframe.DefaultType(series.Float),\n    dataframe.WithTypes(map[string]series.Type{\n        \"A\": series.String,\n        \"D\": series.Bool,\n    }),\n)\n```\n\nSimilarly, you can load the data stored on a `[]map[string]interface{}`:\n\n```go\ndf := dataframe.LoadMaps(\n    []map[string]interface{}{\n        map[string]interface{}{\n            \"A\": \"a\",\n            \"B\": 1,\n            \"C\": true,\n            \"D\": 0,\n        },\n        map[string]interface{}{\n            \"A\": \"b\",\n            \"B\": 2,\n            \"C\": true,\n            \"D\": 0.5,\n        },\n    },\n)\n```\n\nYou can also pass an `io.Reader` to the functions `ReadCSV`/`ReadJSON`\nand it will work as expected given that the data is correct:\n\n```go\ncsvStr := `\nCountry,Date,Age,Amount,Id\n\"United States\",2012-02-01,50,112.1,01234\n\"United States\",2012-02-01,32,321.31,54320\n\"United Kingdom\",2012-02-01,17,18.2,12345\n\"United States\",2012-02-01,32,321.31,54320\n\"United Kingdom\",2012-02-01,NA,18.2,12345\n\"United States\",2012-02-01,32,321.31,54320\n\"United States\",2012-02-01,32,321.31,54320\nSpain,2012-02-01,66,555.42,00241\n`\ndf := dataframe.ReadCSV(strings.NewReader(csvStr))\n```\n\n```go\njsonStr := `[{\"COL.2\":1,\"COL.3\":3},{\"COL.1\":5,\"COL.2\":2,\"COL.3\":2},{\"COL.1\":6,\"COL.2\":3,\"COL.3\":1}]`\ndf := dataframe.ReadJSON(strings.NewReader(jsonStr))\n```\n\n#### Subsetting\n\nWe can subset our DataFrames with the Subset method. For example if we\nwant the first and third rows we can do the following:\n\n```go\nsub := df.Subset([]int{0, 2})\n```\n\n#### Column selection\n\nIf instead of subsetting the rows we want to select specific columns,\nby an index or column name:\n\n```go\nsel1 := df.Select([]int{0, 2})\nsel2 := df.Select([]string{\"A\", \"C\"})\n```\n\n#### Updating values\n\nIn order to update the values of a DataFrame we can use the Set\nmethod:\n\n```go\ndf2 := df.Set(\n    []int{0, 2},\n    dataframe.LoadRecords(\n        [][]string{\n            []string{\"A\", \"B\", \"C\", \"D\"},\n            []string{\"b\", \"4\", \"6.0\", \"true\"},\n            []string{\"c\", \"3\", \"6.0\", \"false\"},\n        },\n    ),\n)\n```\n\n#### Filtering\n\nFor more complex row subsetting we can use the Filter method. For\nexample, if we want the rows where the column \"A\" is equal to \"a\" or\ncolumn \"B\" is greater than 4:\n\n```go\nfil := df.Filter(\n    dataframe.F{\"A\", series.Eq, \"a\"},\n    dataframe.F{\"B\", series.Greater, 4},\n)\n\nfilAlt := df.FilterAggregation(\n    dataframe.Or,\n    dataframe.F{\"A\", series.Eq, \"a\"},\n    dataframe.F{\"B\", series.Greater, 4},\n) \n```\n\nFilters inside Filter are combined as OR operations, alternatively we can use `df.FilterAggragation` with `dataframe.Or`.\n\nIf we want to combine filters with AND operations, we can use `df.FilterAggregation` with `dataframe.And`.\n\n```go\nfil := df.FilterAggregation(\n    dataframe.And, \n    dataframe.F{\"A\", series.Eq, \"a\"},\n    dataframe.F{\"D\", series.Eq, true},\n)\n```\n\nTo combine AND and OR operations, we can use chaining of filters.\n\n```go\n// combine filters with OR\nfil := df.Filter(\n    dataframe.F{\"A\", series.Eq, \"a\"},\n    dataframe.F{\"B\", series.Greater, 4},\n)\n// apply AND for fil and fil2\nfil2 := fil.Filter(\n    dataframe.F{\"D\", series.Eq, true},\n)\n```\n\nFiltering is based on predefined comparison operators: \n* `series.Eq`\n* `series.Neq`\n* `series.Greater`\n* `series.GreaterEq`\n* `series.Less`\n* `series.LessEq`\n* `series.In`\n\nHowever, if these filter operations are not sufficient, we can use user-defined comparators.\nWe use `series.CompFunc` and a user-defined function with the signature `func(series.Element) bool` to provide user-defined filters to `df.Filter` and `df.FilterAggregation`.\n\n```go\nhasPrefix := func(prefix string) func(el series.Element) bool {\n        return func (el series.Element) bool {\n            if el.Type() == String {\n                if val, ok := el.Val().(string); ok {\n                    return strings.HasPrefix(val, prefix)\n                }\n            }\n            return false\n        }\n    }\n\nfil := df.Filter(\n    dataframe.F{\"A\", series.CompFunc, hasPrefix(\"aa\")},\n)\n```\n\nThis example filters rows based on whether they have a cell value starting with `\"aa\"` in column `\"A\"`.\n\n#### GroupBy \u0026\u0026 Aggregation\n\nGroupBy \u0026\u0026 Aggregation\n\n```go\ngroups := df.GroupBy(\"key1\", \"key2\") // Group by column \"key1\", and column \"key2\" \naggre := groups.Aggregation([]AggregationType{Aggregation_MAX, Aggregation_MIN}, []string{\"values\", \"values2\"}) // Maximum value in column \"values\",  Minimum value in column \"values2\"\n```\n\n#### Arrange\n\nWith Arrange a DataFrame can be sorted by the given column names:\n\n```go\nsorted := df.Arrange(\n    dataframe.Sort(\"A\"),    // Sort in ascending order\n    dataframe.RevSort(\"B\"), // Sort in descending order\n)\n```\n\n#### Mutate\n\nIf we want to modify a column or add one based on a given Series at\nthe end we can use the Mutate method:\n\n```go\n// Change column C with a new one\nmut := df.Mutate(\n    series.New([]string{\"a\", \"b\", \"c\", \"d\"}, series.String, \"C\"),\n)\n// Add a new column E\nmut2 := df.Mutate(\n    series.New([]string{\"a\", \"b\", \"c\", \"d\"}, series.String, \"E\"),\n)\n```\n\n#### Joins\n\nDifferent Join operations are supported (`InnerJoin`, `LeftJoin`,\n`RightJoin`, `CrossJoin`). In order to use these methods you have to\nspecify which are the keys to be used for joining the DataFrames:\n\n```go\ndf := dataframe.LoadRecords(\n    [][]string{\n        []string{\"A\", \"B\", \"C\", \"D\"},\n        []string{\"a\", \"4\", \"5.1\", \"true\"},\n        []string{\"k\", \"5\", \"7.0\", \"true\"},\n        []string{\"k\", \"4\", \"6.0\", \"true\"},\n        []string{\"a\", \"2\", \"7.1\", \"false\"},\n    },\n)\ndf2 := dataframe.LoadRecords(\n    [][]string{\n        []string{\"A\", \"F\", \"D\"},\n        []string{\"1\", \"1\", \"true\"},\n        []string{\"4\", \"2\", \"false\"},\n        []string{\"2\", \"8\", \"false\"},\n        []string{\"5\", \"9\", \"false\"},\n    },\n)\njoin := df.InnerJoin(df2, \"D\")\n```\n\n#### Function application\n\nFunctions can be applied to the rows or columns of a DataFrame,\ncasting the types as necessary:\n\n```go\nmean := func(s series.Series) series.Series {\n    floats := s.Float()\n    sum := 0.0\n    for _, f := range floats {\n        sum += f\n    }\n    return series.Floats(sum / float64(len(floats)))\n}\ndf.Capply(mean)\ndf.Rapply(mean)\n```\n\n#### Chaining operations\n\nDataFrames support a number of methods for wrangling the data,\nfiltering, subsetting, selecting columns, adding new columns or\nmodifying existing ones. All these methods can be chained one after\nanother and at the end of the procedure check if there has been any\nerrors by the DataFrame Err field. If any of the methods in the chain\nreturns an error, the remaining operations on the chain will become\na no-op.\n\n```go\na = a.Rename(\"Origin\", \"Country\").\n    Filter(dataframe.F{\"Age\", \"\u003c\", 50}).\n    Filter(dataframe.F{\"Origin\", \"==\", \"United States\"}).\n    Select(\"Id\", \"Origin\", \"Date\").\n    Subset([]int{1, 3})\nif a.Err != nil {\n    log.Fatal(\"Oh noes!\")\n}\n```\n\n#### Print to console\n\n```go\nfmt.Println(flights)\n\n\u003e [336776x20] DataFrame\n\u003e \n\u003e     X0    year  month day   dep_time sched_dep_time dep_delay arr_time ...\n\u003e  0: 1     2013  1     1     517      515            2         830      ...\n\u003e  1: 2     2013  1     1     533      529            4         850      ...\n\u003e  2: 3     2013  1     1     542      540            2         923      ...\n\u003e  3: 4     2013  1     1     544      545            -1        1004     ...\n\u003e  4: 5     2013  1     1     554      600            -6        812      ...\n\u003e  5: 6     2013  1     1     554      558            -4        740      ...\n\u003e  6: 7     2013  1     1     555      600            -5        913      ...\n\u003e  7: 8     2013  1     1     557      600            -3        709      ...\n\u003e  8: 9     2013  1     1     557      600            -3        838      ...\n\u003e  9: 10    2013  1     1     558      600            -2        753      ...\n\u003e     ...   ...   ...   ...   ...      ...            ...       ...      ...\n\u003e     \u003cint\u003e \u003cint\u003e \u003cint\u003e \u003cint\u003e \u003cint\u003e    \u003cint\u003e          \u003cint\u003e     \u003cint\u003e    ...\n\u003e \n\u003e Not Showing: sched_arr_time \u003cint\u003e, arr_delay \u003cint\u003e, carrier \u003cstring\u003e, flight \u003cint\u003e,\n\u003e tailnum \u003cstring\u003e, origin \u003cstring\u003e, dest \u003cstring\u003e, air_time \u003cint\u003e, distance \u003cint\u003e, hour \u003cint\u003e,\n\u003e minute \u003cint\u003e, time_hour \u003cstring\u003e\n```\n\n#### Interfacing with gonum\n\nA `gonum/mat.Matrix` or any object that implements the `dataframe.Matrix`\ninterface can be loaded as a `DataFrame` by using the `LoadMatrix()` method. If\none wants to convert a `DataFrame` to a `mat.Matrix` it is necessary to create\nthe necessary structs and method implementations. Since a `DataFrame` already\nimplements the `Dims() (r, c int)` method, only implementations for the `At` and\n`T` methods are necessary:\n\n```go\ntype matrix struct {\n\tdataframe.DataFrame\n}\n\nfunc (m matrix) At(i, j int) float64 {\n\treturn m.Elem(i, j).Float()\n}\n\nfunc (m matrix) T() mat.Matrix {\n\treturn mat.Transpose{m}\n}\n```\n\nSeries\n------\n\nSeries are essentially vectors of elements of the same type with\nsupport for missing values. Series are the building blocks for\nDataFrame columns.\n\nFour types are currently supported:\n\n```go\nInt\nFloat\nString\nBool\n```\n\nFor more information about the API, make sure to check:\n\n- [dataframe godoc][3]\n- [series godoc][4]\n\nLicense\n-------\nCopyright 2016 Alejandro Sanchez Brotons\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you\nmay not use this file except in compliance with the License.  You may\nobtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\nimplied. See the License for the specific language governing\npermissions and limitations under the License.\n\n[1]: https://github.com/gonum\n[2]: https://github.com/go-gota/gota\n[3]: https://godoc.org/github.com/go-gota/gota/dataframe\n[4]: https://godoc.org/github.com/go-gota/gota/series\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgo-gota%2Fgota","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgo-gota%2Fgota","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgo-gota%2Fgota/lists"}