{"id":19068821,"url":"https://github.com/cutenode/encites","last_synced_at":"2025-08-21T18:33:03.257Z","repository":{"id":46762751,"uuid":"347181241","full_name":"cutenode/encites","owner":"cutenode","description":"Fetches GitHub Activtiy, providing an API for that activity and rendering it prettily in Markdown","archived":false,"fork":false,"pushed_at":"2024-11-20T20:32:06.000Z","size":524,"stargazers_count":39,"open_issues_count":8,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-12-13T02:20:59.687Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cutenode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-12T19:50:37.000Z","updated_at":"2024-02-06T10:06:30.000Z","dependencies_parsed_at":"2022-08-12T13:01:14.478Z","dependency_job_id":null,"html_url":"https://github.com/cutenode/encites","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cutenode%2Fencites","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cutenode%2Fencites/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cutenode%2Fencites/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cutenode%2Fencites/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cutenode","download_url":"https://codeload.github.com/cutenode/encites/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230527866,"owners_count":18240051,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T01:12:08.872Z","updated_at":"2024-12-20T03:09:16.334Z","avatar_url":"https://github.com/cutenode.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Encites\n\nEncites is a WIP project to enable folks to collect and process public GitHub activity data about users. Sorry if anything's broken, I might fix it if you create an issue.\n\n## Usage\n\n```bash\nnpm install encites\n```\n\n### Preface\n\nThere are a few things you'll need:\n\n- Data directory: You'll need a data directory. Regardless of what you name this directory, almost all operations of encites assume you have one. Encites will try to create one for you if one doesn't already exist when you're running some commands, but does not gaurantee that it'll succeed.\n- Megafile name: Since GitHub only returns 30 days or 300 results (whichever is less), Encites provides some functionality that can help you build a cache of all events if you run it often enough (how often is often enough depends on how active any given user is). In the project, this is considered a `megafile` and can be thought of as a kind of cache. To use certain kinds of functionality, you'll need to choose a name for your megafile and keep it consistent. `megafile.json` is a good default.\n\nSome additional terms and context around them:\n\n- Events: Events are the primordial that Encites gets from GitHub. Encites manipulates the Event objects it gets from GitHub through `getPublicGitHubEvents`, and can use `filterPublicGitHubEvents` to be more compact, only surfacing information that are useful for accomplishing the goals of Encites.\n- Events Array: An array that contains individual Events.\n- Events File: A file that contains an Events Array. Generally, written by one of the helper methods.\n- Data Directory: The directory where your JSON output files go.\n\n### Environment Variables\n\nEncites uses [dotenv](https://www.npmjs.com/package/dotenv) to read environment variables from a `.env` file in your project. If you'd prefer not to use a `.env` file, you can define your environment variables in your operating system through normal methods.\n\n#### `ENCITES_GITHUB_PAT`\n\nThis is expected to be assigned the value of a GitHub Personal Access Token (GitHub PAT, see [GitHub's docs](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) for details on how to create one). This is needed to ensure you don't get rate limited, which will almost certainly happen when using Encites without a PAT.\n\n#### `ENCITES_LOGGER`\n\nIf this is assigned the value of `true`, you will get pretty log output from [pino](https://npm.im/pino). You can use [pino-pretty](http://npm.im/pino-pretty) to prettily view them in your terminal.\n\n### Methods\n\n#### dedupeLocalEventsFilesInDirectory(dataDirectory)\n\nA utility in the pursuit of building megafiles. It will retrun a deduped array of all entires from entries files in the passed directory.\n\n* `dataDirectory` (string, required): path to your data directory to step through for events files (Encites presumes that _all_ files in that data directory are events files).\n\n```js\nconst { dedupeLocalEventsFilesInDirectory } = require('encites')\n\nconst directory = './data/`\n\n// get all data, deduped, from your data directory\nconst dedupedData = await dedupeLocalEventsFilesInDirectory(`${directory}`)\n\nconsole.log(dedupedData)\n```\n#### filterPublicGitHubEvents(events)\n\nParses data from the GitHub API, provided to the method via the `events` property, and returns a nice clean array of objects representing events with only relevant information.\n\n* `events` (array, required): an array of GitHub API response data.\n\n```js\nconst { filterPublicGitHubEvents, getPublicGitHubEvents } = require('encites')\n\nconst arrayOfGitHubUsers = ['bnb']\n\n// get our GitHub data - make sure you've already set ENCITES_GITHUB_PAT\nconst publicEvents = await getPublicGitHubEvents(arrayOfGitHubUsers)\n\n// filter our data into the shape the rest of the module expects\nconst events = await filterPublicGitHubEvents(publicEvents)\n```\n\n#### getEventsFrom\n\n`getLocalEventsFrom()` has two exposed methods - `date` and `period`. These respectively allow you to **filter** events contained within events files, based on date/time information.\n\n##### getEventsFrom.date(eventsFile, startDate)\n\nGets all events, starting after the date passeed.\n\n* `eventsFile` (string, required): Path to an events file. You'll probably want to read a megafile rather than any given single events file. \n* `startDate` (string, required): The starting date. Any events before this date will be excluded from the results. Accepts any value that is valid in Luxon's [.toISO()](https://moment.github.io/luxon/docs/class/src/datetime.js~DateTime.html#instance-method-toISO) method.\n\n```js\nconst { getEventsFrom } = require('encites')\n\nconst dataPath = './data/'\nconst megafileName =  'megafile.json'\nconst users = ['bnb']\n\nconst dateToCheckFrom = '2021-03-01' // yyyy-mm-dd\n\nconst dateFilteredEvents = await getLocalEventsFrom.date(`${dataPath}${megafileName}`, dateToCheckFrom)\n```\n\n##### getEventsFrom.period(eventsFile, startDate, endDate)\n\nGet all events within a given period.\n\n* `eventsFile` (string, required): Path to an events file. You'll probably want to read a megafile rather than any given single events file.\n* `startDate` (string, required): The starting date. Any events before this date will be excluded from the results. Accepts any value that is valid in Luxon's [.toISO()](https://moment.github.io/luxon/docs/class/src/datetime.js~DateTime.html#instance-method-toISO) method.\n* `endDate` (string, required): The ending date. Any events after this date will be excluded from the results. Accepts any value that is valid in Luxon's [.toISO()](https://moment.github.io/luxon/docs/class/src/datetime.js~DateTime.html#instance-method-toISO) method.\n\n\n```js\nconst { getEventsFrom } = require('encites')\n\nconst dataPath = './data/'\nconst megafileName =  'megafile.json'\nconst users = ['bnb']\n\nconst dateToCheckFrom = '2021-03-01' // yyyy-mm-dd\nconst dateToCheckUntil = '2021-03-28' // yyyy-mm-dd\n\nconst dateFilteredEvents = await getLocalEventsFrom.period(`${dataPath}${megafileName}`, dateToCheckFrom, dateToCheckUntil)\n```\n\n#### getMarkdownFromEvents(events, title)\n\nTakes an Events Array, spits out pretty markdown.\n\n* `events` (array, required): An array of Events.\n* `title` (string, required): A string to be used as the title of your Markdown document.\n\n```js\nconst { filterPublicGitHubEvents, getMarkdownFromEvents getPublicGitHubEvents } = require('encites')\n\n// the users we're getting data for\nconst users = ['bnb']\n\n// fetch new data from GitHub - make sure you've already set ENCITES_GITHUB_PAT\nconst publicEvents = await getPublicGitHubEvents(users)\n\n// structure our data in the shape we need\nconst events = await filterPublicGitHubEvents(users)\n\n// get a markdown representation of our Events\nconst markdown = await getMarkdownFromEvents(events)\n``` \n\n#### getPublicGitHubEvents(githubUsernames)\n\nFetches data from the GitHub API. Only fetches public data, zero authenticated/private data is fetched. \n\nYou can either directly pass this along to something that consumes it, or you can write it to a file. Writing it to a file has the benefit of being able to be consumed again later if you'd like to rebuild your data or if there are new Event parsing capabilities added to the module once you're past 90 days or 300 events.\n\n* `githubUsernames` (string, required): An Array of GitHub usernames for whom you want to consume public data.\n\n```js\nconst { filterPublicGitHubEvents, getPublicGitHubEvents } = require('encites')\n\nconst arrayOfGitHubUsers = ['bnb']\n\n// get our GitHub data - make sure you've already set ENCITES_GITHUB_PAT\nconst publicEvents = await getPublicGitHubEvents(arrayOfGitHubUsers)\n\n// filter our data into the shape the rest of the module expects\nconst events = await filterPublicGitHubEvents(publicEvents)\n```\n\n#### writeEventsFile(dataDirectory, events, options)\n\nWrites an Events File to the passed Data Directory with the passed Events Array. By default, the filename is the current date in `yyyy-mm-dd` format, but you can overwrite that with options. Doing so is useful for writing megafiles.\n\n* `dataDirectory` (string, required): the path to your data directory, where an Events File will be written to.\n* `events` (array, required): Pass in an Event Array to be written to the provided path.\n* `options` (object, optional):\n  * `filename` (string, optional): The name of the file. Useful for writing megafiles.\n\n```js\nconst { filterPublicGitHubEvents, getPublicGitHubEvents, writeEventsFile } = require('encites')\n\n// path to write all our files to.\n  const dataPath = './data/'\n\n// array of users we'd like data for\nconst users = ['bnb']\n\nconst publicEvents = await getPublicGitHubEvents(users)\n\n// fetches public data from  the GitHub API\nconst events = await filterPublicGitHubEvents(publicEvents)\n\n// write single instance of data\nwriteEventsFile(`${dataPath}`, events)\n```\n\n#### writeMarkdownFile(markdownPath, markdownFileName, events, title)\n\nA relatively straightforward wrapper of [`getMarkdownFromEvents()`](#getmarkdownfromeventsevents) to write events out as a pretty Markdown file to a provided path with a provided filename.\n\n- `markdownPath` (string, required): The _full path_ to which you want the markdown file to be written. Does not include filename.\n- `markdownFileName` (string, required): The filename, including extension, that you want your resulting markdown to be written as.\n- `events` (array, required): An array of Events that you'd like to be parsed out into Markdown.\n- `title` (string, required): A string to be used as the title of your Markdown document.\n\n```js\nconst { filterPublicGitHubEvents, getPublicGitHubEvents, writeMarkdownFile } = require('encites')\n\n// in this case, we're fine with our markdown file being in the root \nconst markdownPath = './'\nconst markdownFilename = 'output.md'\n\n// the usernames we want public data for\nconst users = ['bnb']\n\n// fetch the public data - make sure you've already set ENCITES_GITHUB_PAT\nconst publicEvents = await getPublicGitHubEvents(users)\n\n// fetches public data from  the GitHub API\nconst events = await getAndFilterPublicGitHubEvents(publicEvents)\n\n// write single instance of data\nwriteMarkdownFile(markdownPath, markdownFileName, events)\n```\n#### writemegafile(dataPath, megafileFileName)\n\nA tiny wrapper over [`writeEventsFile()`](#writeeventsfiledatadirectory-events-options) to simplify writing megafiles. Writes a megafile to the data directory.\n\n- `dataPath` (string, required): the path to your data directory. \n- `megafileFileName` (string, required): the name of your megafile. `megafile.json` is a good choice.\n\n```js\nconst { writeMegafile } = require('encites')\n\n// data directory that we write all our files to\nconst dataPath = './data/'\n\n// name of the megafile. Can be whatever, I've just chosen megafile. Needs to be `.json`.\nconst megafileName =  'megafile.json'\n\n// write our megafile to the data directory\nwriteMegafile(dataPath, megafileName)\n```\n\n## Appendix A: Object Shapes\n\n### Event Object\n\nAn event object has a relatively specific structure. This structure is based on which `type` of event the Event Object is representing. These structures are built out in `./lib/getAndFilterPublicGitHubEvents.js`. Here's a reference for each kind:\n\n```json\n\n// IssuesEvent\n{\n  \"id\": \"15475051640\",\n  \"type\": \"IssuesEvent\",\n  \"author\": \"bnb\",\n  \"repo\": \"cutenode/delice\",\n  \"title\": \"f\",\n  \"link\": \"https://github.com/cutenode/delice/pull/8\",\n  \"number\": 8,\n  \"date\": \"2021-03-10T00:27:37Z\",\n  \"action\": \"opened\"\n},\n\n// PullRequestEvent\n{\n  \"id\": \"15506478969\",\n  \"type\": \"PullRequestEvent\",\n  \"author\": \"bnb\",\n  \"repo\": \"openjs-foundation/cross-project-council\",\n  \"title\": \"doc: add README.md file to /TRAVEL_FUND\",\n  \"link\": \"https://github.com/openjs-foundation/cross-project-council/pull/727\",\n  \"number\": 727,\n  \"date\": \"2021-03-11T16:27:38Z\",\n  \"action\": \"opened\"\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcutenode%2Fencites","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcutenode%2Fencites","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcutenode%2Fencites/lists"}