Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with batch-processing

A curated list of projects in awesome lists tagged with batch-processing .

https://github.com/adambirds/chronos

Chronos: A Python library for advanced task timing with features like distributed timing, batch timing, real-time visualization, and debugging-friendly timers.

batch-processing debugging dev-tools developer-tools distributed-timing open-source performance-monitoring profiling python python3 real-time-visualization task-timer task-timing timer

Last synced: 09 Feb 2026

https://github.com/aymendaoudi/evcharging

Simulation, Ingestion and ETL-ing data of millions of EV charging sessions by thousands of EVs in thousands of stations around the world.

batch-processing data-engineering data-lake data-warehouse kafka kafka-connect lake-fs minio mongodb postgresql python3 simpy spark stream-processing

Last synced: 12 Mar 2025

https://github.com/dimmaryanto93/hibernate.orm-example

Belajar Hibernate ORM (Object Relational Mapping)

batch-processing criteria-api hibernate hql

Last synced: 17 Oct 2025

https://github.com/kishorep26/api-doc-verfiy-validate-img-tools

Document processing API with automated resizing, validation, and verification using Google Cloud Vision for intelligent image analysis

api automation batch-processing cloud-api document-management document-validation document-verification flask google-cloud-vision image-processing image-resize python rest-api verification-tool web-aplication

Last synced: 14 Mar 2026

https://github.com/wilddev/images

Reactive images processing service. Works on RabbitMQ and MongoDB

batch-processing crop image-cdn image-service resize

Last synced: 26 Jun 2025

https://github.com/nbkredspy/deeplbatch

A command line tool that leverages DeepL.com to translate individual lines in an input file as separate translations.

batch batch-processing deepl translation

Last synced: 27 Apr 2026

https://github.com/berntpopp/variant-linker

🧬 Powerful CLI tool and JavaScript library for genetic variant annotation using Ensembl APIs. Supports SNVs, CNVs, VCF files, inheritance analysis, batch processing, and genome assembly liftover.

batch-processing bioinformatics cli-tool cnv computational-biology ensembl genetic-variants genetics genome-assembly genomics inheritance-analysis javascript-library liftover medical-genetics nodejs snp variant-annotation variant-calling vcf vep

Last synced: 23 Jan 2026

https://github.com/tvanreenen/batch-url-validator

A simple utility script to validate multiple URLs in parallel.

batch-processing url-validator

Last synced: 25 Jun 2025

https://github.com/iv-mexx/beautifyfileheader

A python tool that unifies and beautifies all source-file-headers in a directory

author batch-processing beautify copyright file-headers python unify

Last synced: 26 Oct 2025

https://github.com/jiatangzhi/master_thesis

This project implements my master’s thesis on building a scalable, ACID-compliant data lakehouse architecture for IoT and industrial workloads, integrating AWS Glue, S3, Athena, and Grafana with Iceberg to evaluate Copy-on-Write vs Merge-on-Read performance.

apache-iceberg aws-glue aws-s3 batch-processing data-engineering data-lakehouse distributed-systems grafana iot-data mqtt open-table-format python3 schema-evolution spark

Last synced: 04 May 2026

https://github.com/aymendaoudi/electric-vehicle-charging-simulator

Simulation, Ingestion and ETL-ing data of millions of EV charging sessions by thousands of EVs in thousands of stations around the world.

apache-airflow apache-spark batch-processing data-engineering data-lake data-warehouse kafka kafka-connect lake-fs minio mongodb postgresql python3 simpy spark stream-processing

Last synced: 15 Apr 2026

https://github.com/thewtfdude/soundcloud-reset-2.0

A userscript to automate the process of removing unwanted Likes, Comments, Reposts. This script can be customized to delete all or a specific number of entries and works seamlessly with the Tampermonkey userscript manager. Ideal for those looking to clean up their SoundCloud profiles quickly and efficiently.

automation batch-processing cleanup comments delete likes reposts soundcloud soundcloud-unrepost tampermonkey userscript

Last synced: 08 May 2026

https://github.com/neerajd12/pybatch

Framework to simplify multi-core processing in python.

batch-processing multiprocessing python python-3

Last synced: 20 Jan 2026

https://github.com/ljrobinson/unitmath

Pure Rust unit conversion engine and CLI for weight, volume, potency, package math, and batch CSV/JSON workflows.

batch-processing cli command-line-tool conversion csv data-cleaning json jsonl math rust unit-conversion units

Last synced: 12 May 2026

https://github.com/jcaperella29/jcap_ai_paper_summarizer

A Flask-based web app that extracts text, summarizes scientific PDFs using Ollama, and retrieves figures from research papers. Supports batch processing via ZIP uploads and provides downloadable summaries.

ai-summarizer batch-processing flask nlp ollama-api pdf-summarizer research-tool scientific-papers

Last synced: 09 May 2026

https://github.com/frobnitzem/psik_api

An API wrapping psik.

api-rest batch-processing hpc

Last synced: 18 Mar 2025

https://github.com/dongryeollee1/openai_asynchronous_use

Openai API Asynchronously Using Script

asyncio batch-processing openai openai-api

Last synced: 17 May 2026

https://github.com/koykov/batch_query

Batch operations abstraction over various storages.

batch-processing system-design

Last synced: 07 Apr 2026

https://github.com/alexwithstars/particlesim

A particle sim with an effect based in particle js home page made from scratch

batch-processing canvas collision-detection javascript js

Last synced: 19 Jul 2025

https://github.com/paultheal1en/convert-pe-file-using-ida-pro-to-asm-and-bytes

A PowerShell script to convert PE (Portable Executable) files into ASM (Assembly) and raw byte code using IDA Pro. Designed for reverse engineering, malware analysis, and security research

assembly-code batch-processing bytecode ida-pro malware-analysis pe-file powershell reverse-engineering security-tools

Last synced: 10 Feb 2026

https://github.com/rbwtech/easy-to-you-automation

Quickly and easily decrypt ioncube encoded PHP scripts. works using easytoyou.eu and handles large directories automatically. essential for working with larger codebases.

automation batch-processing decoder easytoyou encryption ioncube php security web-scraping wordpress

Last synced: 16 Apr 2026

https://github.com/arthur777-br/oltp-simulator

🏥 Simulate realistic hospital data in PostgreSQL with continuous INSERT/UPDATE operations for effective CDC testing with Debezium or other ingestion engines.

batch-processing chinook-database data-engineering data-generator neon neondb postgresql sample-database synthetic-data

Last synced: 16 Apr 2026

https://github.com/manishmlv50/smartqueue

SmartQueue is a lightweight, high-performance .NET queue library built for concurrent processing, CPU-based throttling, and batch workflows. Ideal for scalable background jobs and multi-stage task pipelines. Includes DI support, metrics, and real-world demo.

background-jobs batch-processing dotnet multithreading nuget-package pipelines queues throttling worker

Last synced: 04 Mar 2026

https://github.com/amekusa/feed.sh

Batch-process the lines in a file with an arbitrary command. Supports parallel processing, retrying, intervals, logging, and interactive queueing.

automation bash batch-processing cli devops downloader queue shell

Last synced: 16 Apr 2026

https://github.com/haddouguihamza/banking-balancefileprocessing

Process basic flat files (doc, docx, csv) holding banking records known as balance data (client name, type of payment card, transaction amount, transaction date, real balance, fictious balance ...) using batch process developed with Spring batch and build with Maven

batch-processing batch-programming eclipse-ide flatfile h2-in-memory-database java java-ee maven restful-api spring-batch spring-boot spring-hateoas spring-mvc

Last synced: 14 Jul 2025

https://github.com/pasblinn/universal-converter

🔄 Universal Converter - Professional Windows application for converting documents, images, audio, video, archives, and code files. Features modern GUI, batch processing, and 100% offline operation.

batch-processing converter desktop-app ffmpeg file-converter file-formats nsis-installer pyqt5 python windows

Last synced: 01 May 2026

https://github.com/natsunlee/flowmancer

The Python Thing-Doer

batch batch-processing python task-runner

Last synced: 13 Jul 2025

https://github.com/ghomashudson/pyspark-batch-ai

Batch process Spark DataFrames with LLMs

ai batch batch-processing llms openai pyspark

Last synced: 16 May 2026

https://github.com/star-plan/image-glider

ImageGlider 是一个基于 .NET 8 和 ImageSharp 的跨平台批量图像格式转换工具，支持 AOT 编译。它能自动扫描指定扩展名的图像文件，将其转换为目标格式，并智能管理输出、失败文件及日志记录，为图像转换提供高效便捷的解决方案。

aot batch-processing cli cross-platform csharp dotnet dotnet8 image-conversion image-processing imagesharp

Last synced: 12 Apr 2025

https://github.com/zvdy/system-design

A practical guide to system design patterns with real-world Python implementations.

batch-processing caching databases load-balancer python sharding system-design

Last synced: 03 May 2026

https://github.com/theseraphim/naps2-cli-wrapper

Scanner automation scripts in C# (.NET 8) and Python - CLI wrappers for NAPS2 with auto-detection, named parameters, and multi-format output

automation batch-processing cli cross-platform document-scanning dotnet naps2 python scanner scripts

Last synced: 10 May 2026

https://github.com/dooml/mermaid-bulk-converter

🔄 Convert Mermaid diagram files (.mmd) to high-resolution PNG images with optimized settings. This Python utility provides batch processing capabilities, customizable scaling, and transparent backgrounds, making it perfect for documentation and diagram management workflows.

automation-tool batch-processing cli-tool developer-tools diagram-converter diagram-generation documentation-tool high-resolution-images mermaid mermaid-cli mermaid-diagrams png-converter python

Last synced: 04 Apr 2025

https://github.com/johnathon-smith/aws-glue-snowflake-dbt-pipeline

Modern batch data pipeline using AWS Glue, S3, Snowflake, and dbt. Showcases secure ingestion, external stages, layered transformations, dbt testing, and production deployment with environment separation.

amazon-s3 aws aws-glue batch-processing data-engineering data-pipeline data-warehouse dbt etl iam production-pipeline snowflake

Last synced: 17 Apr 2026

https://github.com/lemonsoda9013/easy-watermark-tool

A batch image watermarking tool based on Python and Tkinter.

batch-processing gui image-processing numpy pillow python tkinter watermark

Last synced: 15 May 2026

https://github.com/prodigysn95/universal-file-converter

Universal File Converter is a Flask web app that converts images, videos, and audio with multi-file upload, real-time progress, and ZIP downloads. 🐙

audio-converter batch-processing batch-processing-of-multiple-files chordpro church churchtech converter ffmpeg file-converter flask image-converter opensong parser parser-library pro5 rhythm-game songpro songshowplus

Last synced: 17 Apr 2026

https://github.com/e2m32/handbrake-batch-transcoder-win

Batch transcode directories on Windows with HandBrakeCLI; multi-threaded workers, interactive pause menu, intelligent skipping, and detailed logging.

batch-processing ffmpeg handbrake handbrake-cli handbrake-script handbrakecli multithreading python python3 transcoding video windows windows-10 windows-11

Last synced: 18 Apr 2026

https://github.com/biyachuev/yt-transcriber

AI-powered audio/video processing: transcription, speaker diarization, LLM refinement, translation | AI-обработка аудио/видео: транскрибация, распознавание спикеров, перевод

ai-pipeline audio-intelligence audio-processing batch-processing document-generation llm media-processing nllb nlp ollama openai-api speaker-diarization speech-to-text translation video-transcription voice-activity-detection whisper youtube-downloader

Last synced: 16 May 2026

https://github.com/dwain-barnes/llm-gguf-auto-converter

Automated Jupyter notebook solution for batch converting Large Language Models to GGUF format with multiple quantization options. Built on llama.cpp with HuggingFace integration.

auto-converter batch-processing cuda gguf huggingface jupyter-notebook llama-cpp llm model-quantization

Last synced: 17 Jun 2025

https://github.com/decepticon-ts/cap-ai-studio

Description: A modern, powerful web application for advanced image analysis and batch processing, featuring real-time AI-powered image captioning, comprehensive reporting, and an intuitive user interface. Built with Streamlit and Google's Gemini API.

artificial-intelligence batch-processing computer-vision data-analysis gemini-api image-processing image-processing-python python streamlit streamlit-webapp threading

Last synced: 19 Apr 2026

https://github.com/itsteknas/pooled-batch-processor

A utility wrapper around es6-promise-pool that lets you process data arrays in batches

batch-processing node node-module pooling

Last synced: 19 Apr 2026

https://github.com/bruce-mig/spring-batch

This repository demonstrates the process of batch file polling and upload through batch processing.

apache-kafka batch-processing file-polling spring-batch-integration

Last synced: 20 Apr 2026

https://github.com/minecraft-gilde/importer

ETL/Importer für Minecraft-Gilde.de: importiert Minecraft-Stats (world/stats) in MariaDB und baut Leaderboards; inkl. Mojang Name-Resolver.

batch-processing cli cron data-pipeline etl importer mariadb minecraft minecraft-stats mojang-api python stream-processing

Last synced: 20 Apr 2026

https://github.com/jollyboss123/spring-batch

This repo sought to provide auto configuration for Spring Batch and remote chunking

batch-processing spring-batch

Last synced: 09 Sep 2025

https://github.com/nyov/postgres-bindata-batchimport

PostgreSQL batch-importing binary files, from disk to database table. Using the Large Objects interface.

batch-processing command-line data-import large-object postgresql psql shell-script

Last synced: 27 Feb 2025

https://github.com/barandev/bulk-image-optimizer

A user-friendly bulk image optimization tool with a customizable GUI that supports multiple formats, resizing, and format conversion.

batch-processing desktop-application gui image-compression image-optimization image-resizing pillow python tkinter

Last synced: 10 Mar 2025

https://github.com/ygalaxyy/cardora-ai-automation

🚀 Cardora AI — Промышленная система автоматизации карточек товаров для Wildberries, Ozon и Яндекс Маркета. Нейросетевой апскейл (Real-ESRGAN), массовая генерация инфографики через Photoshop и умная интеграция с Excel. Масштабируйте контент-продакшн без рутины.

automation batch-processing commercial-products computer-vision content-factory enterprise-software excel-automation image-restoration infographics nuitka ozon photoshop-scripting pyside6 python3 qt6 real-esrgan retail-tech smart-automation wildberries yandex-market

Last synced: 01 Apr 2026

https://github.com/willsoss/jobs

A collection of Runly jobs I use at home.

batch-job batch-processing jobs processes

Last synced: 10 Jun 2025

https://github.com/saenai255/pbatch

A lightweight Go package for processing a slice of items concurrently in batches.

batch batch-processing channels go golang goroutine lightweight

Last synced: 24 Mar 2025

https://github.com/poacosta/prestashop-api-search-tool

High-performance multithreaded Python utility for validating Prestashop product search functionality at scale through systematic API interrogation and reference verification.

batch-processing prestashop-api python search

Last synced: 24 Mar 2025

https://github.com/btskinner/batch_reports

An example of how batch process reports with Bash and knitr

bash batch-processing knitr r reports

Last synced: 11 Apr 2026

https://github.com/juniorvieira99/jr_writer

A component for a large logging system that deal with concurrency and connection pooling for file writing.

batch-processing concurrency connection-pooling context exponential-backoff file-management go golang io performance retry test-driven-development thread-pool thread-safe threading timeout writers

Last synced: 25 Mar 2025

https://github.com/cataerogong/batcher

对于支持命令行参数的程序，用不同的 差异参数 来多次运行程序

autohotkey batch batch-processing windows

Last synced: 31 Mar 2025

https://github.com/mawngo/batch4j

Simple batch processing library for java

batch-processing java

Last synced: 28 Oct 2025

https://github.com/rstoetter/csqlscript-php

a PHP class to execute SQL scripts (mysql)

batch-processing mysql parser php5 script-sql sql

Last synced: 04 May 2026

https://github.com/kiarashyavari/avg_close_price

Scenario: You have CSV files containing daily stock prices with columns: Date, Ticker, Open, High, Low, Close, and Volume. You want to compute the daily average closing price for each stock.

batch-processing pyspark python3 spark sql

Last synced: 11 Mar 2026

https://github.com/terilios/smartsheet-server

MCP server providing healthcare analytics capabilities for Smartsheet, including clinical note summarization, patient feedback analysis, and research impact assessment

azure-openai batch-processing clinical-research healthcare-analytics healthcare-automation mcp-server medical-research patient-feedback smartsheet

Last synced: 27 Mar 2025

https://github.com/vladinator/sharepoint-batch

Javascript library for executing SharePoint REST requests against $batch.

batch batch-processing batching javascript javascript-library rest sharepoint sharepoint-2016 sharepoint-online typescript

Last synced: 08 May 2026

https://github.com/rilltome/universal-file-converter

A Flask-based web app to convert images, videos, and audio files into multiple formats with batch support, ZIP download, and real-time progress.

audio-converter batch-processing batch-processing-of-multiple-files ffmpeg file-converter flask image-converter multi-format python video-converter web-app

Last synced: 11 Mar 2026

https://github.com/yylego/egobatch

Go generic batch task processing with type-safe custom handling

batch-processing concurrent-tasks egobatch go-generics

Last synced: 11 Mar 2026

https://github.com/soerenschneider/batch-nfpm

Allows batch building multiple releases using NFPM

batch-processing nfpm release-automation releases

Last synced: 23 Jul 2025

https://github.com/danielsan80/jobboy-api

A module for integrate JobBoy controllers and routes in a Symfony app

api batch-processing jobboy packagist

Last synced: 21 Apr 2026

https://github.com/magifd2/llm-batch

A helper tool for llm-cli to batch-process JSON/JSONL data, featuring sequential/parallel execution and streaming.

ai batch-processing cli command-line-tool developer-tools go golang json jsonl llm llm-cli

Last synced: 14 May 2026

https://github.com/romanow/batch-migration

Large data migration with Spring Batch

batch-processing spring-boot

Last synced: 19 Apr 2026

https://github.com/jacobeliasmoya/productivepython

CLI tools for batch geotagging and image compression. GPS tagging by folder name, size-based compression, WebP conversion, and optional .exe builds.

batch-processing cli-tool geotagging image-compression image-metadata image-opti python

Last synced: 21 Feb 2026

https://github.com/deniedaccesslife/directoryarchiver

Program to automatically archive and encrypt directories using 7-Zip, saving passwords to a log file and deleting processed source folders.

7-zip archive archiving automation backup batch-processing compression csharp data-protection decryption directory dotnet encryption management password password-generator password-manager security winrar

Last synced: 01 May 2026

https://github.com/geo-mena/squish

🎴 Squish - Batch Browser-based Image Compression

batch batch-processing image-compression image-processing

Last synced: 15 Mar 2025

https://github.com/ramesh-datastack/upramesh-dataengineering-projects

A full-stack data engineering portfolio project with ingestion, batch processing, star schema modeling, orchestration, and analytics dashboard.

analytics batch-processing data-engineering duckdb ecommerce etl-automation etl-pipeline portfolio-project python python3 reports-dashboard star-schema streamlit

Last synced: 07 May 2026

https://github.com/liuyupenggiamlesfsefsfsefes/celflow-batch-excel-csv

🚀 Celflow - 一个高效、轻量的命令行工具，用于 Excel 和 CSV 的批量处理，支持转换、合并、去重等功能。 A fast & lightweight CLI tool for batch processing Excel & CSV data, supporting conversion, merging, deduplication & more.

automation batch-processing convert csv excel java-8

Last synced: 08 Mar 2025