{"id":13425999,"url":"https://github.com/openforis/sepal","last_synced_at":"2026-01-16T17:23:59.400Z","repository":{"id":37852029,"uuid":"38610519","full_name":"openforis/sepal","owner":"openforis","description":"Geographical Data Processing in the Cloud","archived":false,"fork":false,"pushed_at":"2024-10-29T09:20:41.000Z","size":156891,"stargazers_count":210,"open_issues_count":90,"forks_count":46,"subscribers_count":36,"default_branch":"master","last_synced_at":"2024-10-29T11:38:07.609Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://sepal.io/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openforis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"license.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-07-06T09:47:47.000Z","updated_at":"2024-10-29T09:20:45.000Z","dependencies_parsed_at":"2023-09-28T10:58:15.127Z","dependency_job_id":"30411bd0-cbe6-4e37-960b-63e397913a19","html_url":"https://github.com/openforis/sepal","commit_stats":{"total_commits":10517,"total_committers":15,"mean_commits":701.1333333333333,"dds":0.4956736711990112,"last_synced_commit":"983f6ce8fca318ec93d852459044fd5647d95909"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openforis%2Fsepal","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openforis%2Fsepal/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openforis%2Fsepal/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openforis%2Fsepal/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openforis","download_url":"https://codeload.github.com/openforis/sepal/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234207739,"owners_count":18796301,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T00:01:23.908Z","updated_at":"2025-03-15T20:31:37.672Z","avatar_url":"https://github.com/openforis.png","language":"JavaScript","readme":"![banner](https://raw.githubusercontent.com/openforis/sepal-doc/master/docs/source/_images/sepal_header.png)\n\nSEPAL\n=====\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/openforis/sepal/blob/master/license.txt)\n[![Documentation Status](https://readthedocs.org/projects/sepal-doc/badge/?version=latest)](https://sepal-doc.readthedocs.io/en/latest/?badge=latest) \n[![Crowdin](https://badges.crowdin.net/sepal/localized.svg)](https://crowdin.com/project/sepal)\n\nSEPAL is a cloud computing platform for geographical data processing. It enables users to quickly process large amount\nof data without high network bandwidth requirements or need to invest in high-performance computing infrastructure.\n\n--------------------------------------------------------------------------------\n\nCurrently available in the following languages:\n\n| English | Français | Español |\n|---------|----------|---------|\n\nYou can contribute to the translation effort on our [crowdin project](https://crowdin.com/project/sepal).\n\n--------------------------------------------------------------------------------\n\nBackground\n----------\nReducing Emissions from Deforestation and Forest Degradation (REDD) is an effort to create a financial value for the\ncarbon stored in forests, offering incentives for developing countries to reduce emissions from forested lands and\ninvest in low-carbon paths to sustainable development. \"REDD+\" goes beyond deforestation and forest degradation,\nand includes the role of conservation, sustainable management of forests and enhancement of forest carbon stocks. The\nUN-REDD Programme is the United Nations collaborative initiative on Reducing Emissions from Deforestation and forest\nDegradation (REDD) in developing countries.\n\n[FAO](http://www.fao.org/home/en/), as a member of the [UN-REDD Programme](http://www.un-redd.org/), is responsible for\nassisting countries in developing robust national forest\nmonitoring systems (NFMS) and operational satellite land monitoring systems (SLMS) to help them to meet the measurement,\nreporting and verification  (MRV) requirements of the REDD+.  Furthermore, countries need help in establishing and\nmaintaining  a SLMS capable of producing the information required to make consequential decisions about forest\nmanagement; decisions that promote sustainable forest management and can potentially mitigate the effects of global\nclimate change on society.  Specifically, a solution is needed to address the existing challenges countries face when\ndeveloping forest monitoring systems, due to difficulties accessing and processing remotely sensed data; a key source\nof information for monitoring forest area and forest area change over large, often remote areas.\n\nTo tackle the problems mentioned above, FAO and Norway are collaborating on the System for Earth Observation\nData Access, Processing and Analysis for Land Monitoring (SEPAL).\n\nIt  consists of the following components:\n\n1. A powerful, cloud-based computing platform for big data processing and storage.\n2. Facilitated access to remote sensing data sources through a direct-access-interface to earth observation data\n   repositories.\n3. A set of open-source software tools, capable of efficient data processing\n4. Related capacity development and technology transfer activities\n\nThe computing platform enables FAO national partners to process data quickly without locally maintained high\nperformance computing infrastructures.  The direct link to data repositories allows fast access to satellite\nimagery and other earth observation data for processing.  The software tools, such as FAO’s\n[Open Foris Geospatial Toolkit](http://www.openforis.org/tools/geospatial-toolkit.html)\nperform powerful image processing, are completely customizable and function similarly ‘on the cloud’ or on the desktop.\n\nScreenshots\n-----------\n![01 landing](https://user-images.githubusercontent.com/149204/132474862-daf724e5-e7f8-4086-9132-c9afde0e6173.png)\n![02 login](https://user-images.githubusercontent.com/149204/132474870-be73899f-f6bb-4d8b-96c5-05bb21a5d53c.png)\n![03 recipe list](https://user-images.githubusercontent.com/149204/132474880-12333a36-dee0-4bdc-a0b4-0e9aab24b601.png)\n![03 create recipe](https://user-images.githubusercontent.com/149204/132481048-6149f776-a7ed-47cb-8f75-3519aa1b8f1e.png)\n![04 optical](https://user-images.githubusercontent.com/149204/132482428-16ef1555-26bc-441a-8717-d65db3b62ef4.png)\n![05 nicfi planet composite](https://user-images.githubusercontent.com/149204/132474895-da433549-5d52-48cf-93ae-23c0ee9d47c0.png)\n![06 sentinel1 time scan](https://user-images.githubusercontent.com/149204/132483174-154e792e-b6ce-4b22-ad08-1b8e4fdda829.png)\n![07 sentinel1 harmonics](https://user-images.githubusercontent.com/149204/132474903-0d1db533-7427-49f6-9981-07aa5a0f6b71.png)\n![08 classification](https://user-images.githubusercontent.com/149204/132474907-d4a018a1-282f-4dbd-b870-90bae470d1a0.png)\n![09 ccdc chart](https://user-images.githubusercontent.com/149204/132474909-3a3c9f9d-4fb9-42b8-be01-2b354c7283a3.png)\n![10 visparams](https://user-images.githubusercontent.com/149204/132474911-13fdd36a-e4fd-4ad2-93e2-e0a53510b1dc.png)\n![11 layers layout](https://user-images.githubusercontent.com/149204/132478296-627a62cd-9d7b-40cf-a1aa-034c50664cf6.png)\n![12 iPhone](https://user-images.githubusercontent.com/149204/132478926-2bf51235-de16-4a11-9bfb-4960b1e5471a.png)\n![13 terminal](https://user-images.githubusercontent.com/149204/132491822-db82fe79-154f-4f60-b0bc-b5a57006c5a4.png)\n![14 apps](https://user-images.githubusercontent.com/149204/132491851-5ac0303f-1064-4e12-9627-f34e3f78d880.png)\n\nArchitectural overview\n----------------------\nThe core of the system is the _SEPAL server_ and the _user sandboxes_. SEPAL server provides a web-based user-interface,\nwhere geospatial data from multiple providers can be searched, processing chains composed and executed, and geospatial\ndata products visualized.\n\nThe user sandboxes are spaces where users get access to a number of geospatial data processing tools, such as those\nincluded in Open Foris Geospatial Toolkit and Orfeo Toolbox, and their own dedicated storage. SEPAL provides users SSH\naccess to their respective sandbox. This can either be done directly with an SSH client, or through a provided web-based\nterminal. Web-based sandbox tools can be accessed over HTTP.\n\nSandboxes are implemented as Docker containers, which in addition to providing isolation between users, allows for very\nflexible deployment. Sandboxes are started when needed, and stopped when not used. This enables them to be deployed in a\ncluster of worker server instances, which can be dynamically scaled up and down based on demand.\n\n### Default AWS deployment\nThere are three types of server instances:\n\n1. SEPAL servers, constantly running, one in each region where SEPAL is deployed. In addition to the features\n   described above, they also are the entry points for user sandboxes. These instances can be\n   fairly small and cheap, and don’t require much storage.\n\n2. Worker instances, running user sandboxes and retrieving data (Landsat, Sentinel etc.). These instances are\n   automatically launched when users access their sandboxes, and terminated when users disconnect. Users get to decide\n   which instance type each sandbox session will be running on.\n\n3. Operation server, one single instance. It tests and deploys the software, monitors the health of the system,\n   and provides a user interface where user usage can be monitored, and disk/instance use quotas can be configured.\n   This instance can be fairly small.\n\nUsers can at times require a lot of processing power and memory for their processing jobs. The large instances\nneeded for this type of jobs are quite expensive. For instance, an r3.8xlarge (32 CPUs, 244 GiB memory) costs over 3 USD\nan hour, which adds up to more than 2,300 USD a month. When using such expensive instances, care have to be taken to\nuse them efficiently, and not have them sitting idle at any time. To maximize the utility of the worker instances,\nSEPAL will automatically launch them when they are requested, and terminate them when they're not used anymore.\nFor instance, if a user run a 10 hours processing job on an r3.8xlarge, the total cost would be 30 USD, with no\nmoney spent on an idling instance.\n\nTo limit the cost of operating SEPAL, each user has a configurable monthly budget to spend on sandbox sessions. For\ninstance, given a monthly budget of 100 USD a user might have used 32 hours of r3.8xlarge, or 250 hours r3.large and 450\nhours t2.large.\n\nAnother costly component is storage, where 1TB of EFS storage costs 300 USD a month. To limit storage costs, each user\nhave configurable disk quota.\n\n### Components and services part of a SEPAL deployment\n\n**HAProxy** -\nOff-the-shelf load balancer, allowing SEPAL to be clustered for availability. Run both SSH and HTTPS on port 443,\nto prevent firewalls from blocking SSH.\n\n**nginx** -\nOff-the-shelf HTTP and reverse proxy server, proxying all SEPAL HTTP endpoints.\n\n**Xterm.js** -\nOff-the-shelf web-based SSH client. Gives  users SSH access to their Sandbox in a web browser.\n\n**Sepal server** -\nProvides the system user interface.\n\n**Data provider** -\nService retrieving geospatial data from various external data providers.\n\n**Sandbox lifecycle manager** -\nService managing the user sandboxes. It deploys them on demand when users requests access, and un-deploys them as soon\nas a user disconnects from them.\n\n**Sandbox SSH gateway** -\nService responsible for dynamically tunnelling SSH connections to users sandbox, while notifying the sandbox lifecycle\nmanager on connects and disconnects.\n\n**Sandbox web proxy** -\nService proxying HTTP connections to user sandboxes. It maintains HTTP sessions, and notifies the sandbox lifecycle\nmanager on session creation and expiry.\n\n**Sandbox** -\nThe user sandboxes are spaces where users get access to a number of geospatial data processing tools. See table below\nfor provided tools.\n\n![SEPAL components](https://raw.githubusercontent.com/openforis/sepal/master/docs/Components.png)\n\n### Software deployed on each users sandbox:\n\n**Open Foris Geospatial Toolkit** -\nA collection of command-line utilities for processing of geospatial data.\n\n**GDAL** -\nA translator library for raster and vector geospatial data formats.\n\n**R** -\nLanguage for statistical computing and graphics.\n\n**RStudio Server** -\nAn IDE for R in a web browser.\n\n**Orfeo ToolBox** -\nLibrary for remote sensing image processing.\n\n**OpenSARKit** -\nTools for Automatic Preprocessing of SAR Imagery.\n\nBuild and Release\n-----------------\nThe project is under active development, and the build and release process is still in flux, so these\ninstructions will change, and improve, over time.\n\n### Prerequisites\nIn order to build and run the SEPAL system, a Linux or macOS installation is needed.\nThe end-users on the other hand, are of course free to use whatever Operating system they prefer, including Windows.\n\nIn addition to this, the following software must be installed:\n\n[Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html),\n[Maven](https://maven.apache.org/download.cgi), and\n[Ansible](http://docs.ansible.com/ansible/intro_installation.html).\nIf you want to run the system locally, you need [Vagrant](https://www.vagrantup.com/downloads.html), and\nto deploy on Amazon Web Services EC2 instances, you need an [AWS account](https://aws.amazon.com/account/).\n\n### Configuration\nTBD\n\n### Build\nTBD\n\n### Deploy\nTBD\n","funding_links":[],"categories":["JavaScript","Observation and Conservation of Ecosystems","Biosphere"],"sub_categories":["Vegitation and Biodiversity","Forest Observation and Management"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenforis%2Fsepal","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenforis%2Fsepal","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenforis%2Fsepal/lists"}