Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/codam-coding-college/fast42-ts
A super fast 42API connector written in typescript 🚀
https://github.com/codam-coding-college/fast42-ts
42api 42born2code typescript
Last synced: 3 days ago
JSON representation
A super fast 42API connector written in typescript 🚀
- Host: GitHub
- URL: https://github.com/codam-coding-college/fast42-ts
- Owner: codam-coding-college
- License: isc
- Created: 2022-08-25T15:32:33.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-24T15:16:23.000Z (4 months ago)
- Last Synced: 2024-10-23T08:33:05.549Z (21 days ago)
- Topics: 42api, 42born2code, typescript
- Language: TypeScript
- Homepage:
- Size: 1.3 MB
- Stars: 14
- Watchers: 1
- Forks: 2
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: CODEOWNERS
Awesome Lists containing this project
README
# Fast42
Fast TS/JS connector to the 42API, for server-side use.
Features:
- Fast! Gets the most out of your rate-limit, so you don't have to wait forever.
- Automatically determines the rate limit of your API key.
- Queues requests (using bottleneck)
- Multi-key support (be carefull, it might be too fast! 🚀)
- Convenience: fetch all pages from an endpoint with a single method call!
- Clustering (v2.1 and up): using Redis you can run multiple instances on the same API keys!
Public Methods:
```ts
constructor(
secrets: ApiSecret[] // Api Secrets, see type below
concurrentOffset?: number, // default is 0, can be used to slow down the requests. ex: if your key can do 4 req/s you can set this to 1 to only make 3 req/s. Usefull if your backend or db can't keep up.
jobExpiration?: number, // default is 20000ms, especially important when using redis to kill infinite jobs
redisConfig?: RedisConfig // config to connect to redis, see below
);
interface ApiSecret {
client_id: string;
client_secret: string;
}
interface RedisConfig {
host: string;
port: number;
password?: string;
}
// Always call .init() first after constructing Fast42!
init(): Promise
getPage(url: string, page: string, options?: {
[key: string]: string;
}): Promise
getAllPages(url: string, options?: {
[key: string]: string;
}, start?: number): Promise[]>
get(endpoint: string, options?: {
[key: string]: string;
}): Promise
delete(endpoint: string): Promise
post(endpoint: string, body: any): Promise
patch(endpoint: string, body: any): Promise
put(endpoint: string, body: any): Promise
// use a user's accesstoken to make the request, you still need to initialize Fast42 with the same api key used to authenticate the user
postWithUserAccessToken(accessToken: AccessToken, endpoint: string, body: any): Promise
// used for testing, just runs a random job on the current limiter
doJob(job: any): Promise;
// Important when using redis! Closes the connection and stops logging.
disconnect(): Promise;
```
### Install
```sh
npm i @codam/fast42
```
Basic usage:
```ts
import Fast42 from "@codam/fast42"
const api = await new Fast42([
{
client_id: "",
client_secret: "",
}
]).init()
const campus_id = 14;
const pages = await api.getAllPages(`/campus/${campus_id}/users`, {
'filter[campus_id]': campus_id.toString(), // this makes no sense but it gives an example of using options
})
```
Obviously your id/secret should come from the environment and not be committed to git. (I recommend using a `.env` file and the `dotenv` package)
How I use it:
```ts
import Fast42, { Response } from "@codam/fast42"
import dotenv from "dotenv";
// utility function for error handling and logging
function getPageNumberFromUrl(url: string): string | undefined {
const match = url.match(/page\[number\]=(\d+)/);
if (match && match[1]) {
return match[1];
}
return undefined;
}
// utility function for logging errors
function printHeaders(headers: any, print: (arg0: string) => void) {
headers.forEach((v: string, k: string) => {
print(`${k}: ${v}`)
})
}
async function getAll42(
api: Fast42,
url: string,
options: { [key: string]: string },
callback: (_: Response) => any,
) {
const pages: Promise[]> = await api.getAllPages(url, options);
console.log(`Retrieving ${pages.length} pages for ${url}`);
// Attach a callback function to be called when the page promise resolves
return Promise.all(pages.map(async (page) => {
let p = await page;
const pagenr = getPageNumberFromUrl(p.url);
// retry when the ratelimit was hit
// (this can happen because the timing on 42api side is different from the timing of the Fast42 ratelimiter)
if (p.status === 429) {
if (pagenr) {
p = await api.getPage(url, pagenr, options);
} else {
console.error(`Failed retry on unkown page for ${url}`);
}
}
if (p.ok) {
console.log(`Recieved ${url} page: ${pagenr}`);
return callback(p);
} else {
printHeaders(p.headers, console.log);
console.error(`Failed to get ${url} page (${p.status}): ${pagenr}`);
}
}));
}
async function getAll42Cursus(api: Fast42) {
return getAll42(api, "/cursus", {}, async (page) => {
(await page.json() as any).forEach(async (c: any) => {
// Insert `c` into DB
})
}).then(async () => {
console.log(`Total: ${/* Cursus count from db*/} Cursi`)
})
}
// Using 2 keys here, but with 8 req/s per key it will might be a bit too fast ;)
async function main() {
const api = await new Fast42([
{
client_id: process.env['FTAPI_UID'],
client_secret: process.env['FTAPI_SECRET'],
},
{
client_id: process.env['FTAPI_UID1'],
client_secret: process.env['FTAPI_SECRET1'],
}
], 1).init()
await getAll42Cursus(api);
}
```
Usage with redis:
```ts
const api = await (new Fast42([
{
client_id: process.env['FTAPI_UID'],
client_secret: process.env['FTAPI_SECRET'],
},
],
0,
20000, // setting an expiration on all jobs is important when clustering!
{
host: "127.0.0.1",
port: 6379,
password: "somepassword"
}).init());
const job = await api.get("/projects/1");
const item = await job.json();
await api.disconnect();
```