https://github.com/ballerina-platform/module-ballerina-ftp
https://github.com/ballerina-platform/module-ballerina-ftp
ballerina ftp hacktoberfest integration sftp wso2
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/ballerina-platform/module-ballerina-ftp
- Owner: ballerina-platform
- License: apache-2.0
- Created: 2017-10-12T08:51:10.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2026-02-20T09:50:59.000Z (5 months ago)
- Last Synced: 2026-02-20T13:44:26.486Z (5 months ago)
- Topics: ballerina, ftp, hacktoberfest, integration, sftp, wso2
- Language: Java
- Homepage:
- Size: 54.5 MB
- Stars: 114
- Watchers: 85
- Forks: 57
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: changelog.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
- awesome-java - Ballerina FTP
README
Ballerina FTP Library
=====================
[](https://github.com/ballerina-platform/module-ballerina-ftp/actions/workflows/build-timestamped-master.yml)
[](https://codecov.io/gh/ballerina-platform/module-ballerina-ftp)
[](https://github.com/ballerina-platform/module-ballerina-ftp/actions/workflows/trivy-scan.yml)
[](https://github.com/ballerina-platform/module-ballerina-ftp/actions/workflows/build-with-bal-test-graalvm.yml)
[](https://github.com/ballerina-platform/module-ballerina-ftp/commits/master)
[](https://github.com/ballerina-platform/ballerina-standard-library/labels/module%2Fftp)
This module provides an FTP/SFTP client and an FTP/SFTP server listener implementation to facilitate an FTP/SFTP connection connected to a remote location. Additionally, it supports FTPS (FTP over SSL/TLS) to facilitate secure file transfers.
### FTP client
The `ftp:Client` connects to an FTP server and performs various operations on the files. Currently, it supports the
generic FTP operations; `get`, `delete`, `put`, `append`, `mkdir`, `rmdir`, `isDirectory`, `rename`, `size`, and
`list`. The client also provides typed data operations for reading and writing files as text, JSON, XML, CSV, and binary data, with streaming support for handling large files efficiently.
An FTP client is defined using the `protocol` and `host` parameters and optionally, the `port` and
`auth`. Authentication configuration can be configured using the `auth` parameter for Basic Auth and
private key.
An authentication-related configuration can be given to the FTP client with the `auth` configuration.
##### Create a client
The following code creates an FTP client and performs the I/O operations, which connect to the FTP server with Basic Auth.
```ballerina
// Define the FTP client configuration.
ftp:ClientConfiguration ftpConfig = {
protocol: ftp:FTP,
host: "",
port: ,
auth: {
credentials: {
username: "",
password: ""
}
}
};
// Create the FTP client.
ftp:Client|ftp:Error ftpClient = new(ftpConfig);
```
##### Create a directory
The following code creates a directory in the remote FTP server.
```ballerina
ftp:Error? mkdirResponse = ftpClient->mkdir("");
```
##### Upload a file to a remote server
The following code uploads a file to a remote FTP server.
```ballerina
stream fileByteStream
= check io:fileReadBlocksAsStream(putFilePath, );
ftp:Error? putResponse = ftpClient->put("", fileByteStream);
```
You can also upload files as specific types:
Upload as text:
```ballerina
ftp:Error? result = ftpClient->putText("", "Hello, World!");
```
Upload as JSON or record:
```ballerina
// Upload JSON
json jsonData = {name: "John", age: 30};
ftp:Error? result = ftpClient->putJson("", jsonData);
// Upload a record
type User record {
string name;
int age;
};
User user = {name: "Jane", age: 25};
ftp:Error? result = ftpClient->putJson("", user);
```
Upload as XML:
```ballerina
xml xmlData = xml `mydb`;
ftp:Error? result = ftpClient->putXml("", xmlData);
```
Upload as CSV (string arrays or typed records):
```ballerina
// Upload as string array of arrays
string[][] csvData = [["Name", "Age"], ["John", "30"], ["Jane", "25"]];
ftp:Error? result = ftpClient->putCsv("", csvData);
// Upload records as CSV
type Person record {
string name;
int age;
};
Person[] people = [{name: "John", age: 30}, {name: "Jane", age: 25}];
ftp:Error? result = ftpClient->putCsv("", people);
```
Upload as bytes:
```ballerina
byte[] binaryData = [0x48, 0x65, 0x6C, 0x6C, 0x6F]; // "Hello"
ftp:Error? result = ftpClient->putBytes("", binaryData);
```
##### Compress and upload a file to a remote server
The following code compresses and uploads a file to a remote FTP server.
```ballerina
// Set the optional boolean flag as 'true' to compress before uploading
stream fileByteStream
= check io:fileReadBlocksAsStream(putFilePath, );
ftp:Error? compressedPutResponse = ftpClient->put("",
fileByteStream, compressionType=ZIP);
```
##### Get the size of a remote file
The following code gets the size of a file in a remote FTP server.
```ballerina
int|ftp:Error sizeResponse = ftpClient->size("");
```
##### Read the content of a remote file
The following code reads the content of a file in a remote FTP server. The FTP client supports various data types including text, JSON, XML, CSV, and binary data through typed get operations.
Read as text:
```ballerina
string fileContent = check ftpClient->getText("");
```
Read as JSON or typed record:
```ballerina
// Read as JSON
json jsonData = check ftpClient->getJson("");
// Read as a specific record type
type User record {
string name;
int age;
};
User userData = check ftpClient->getJson("");
```
Read as XML or typed record:
```ballerina
// Read as XML
xml xmlData = check ftpClient->getXml("");
// Read as a specific record type
type Config record {
string database;
int timeout;
};
Config config = check ftpClient->getXml("");
```
Read as CSV (string arrays or typed records):
```ballerina
// Read as string array of arrays
string[][] csvData = check ftpClient->getCsv("");
// Read as an array of typed records
type CsvRecord record {
string id;
string name;
string email;
};
CsvRecord[] records = check ftpClient->getCsv("");
```
Read as bytes:
```ballerina
// Read entire file as byte array
byte[] fileBytes = check ftpClient->getBytes("");
// Read as streaming bytes
stream byteStream = check ftpClient->getBytesAsStream("");
record {|byte[] value;|} nextBytes = check byteStream.next();
check byteStream.close();
```
##### Rename/move a remote file
The following code renames or moves a file to another location in the same remote FTP server.
```ballerina
ftp:Error? renameResponse = ftpClient->rename("",
"");
```
##### Delete a remote file
The following code deletes a remote file in a remote FTP server.
```ballerina
ftp:Error? deleteResponse = ftpClient->delete("");
```
##### Remove a directory from a remote server
The following code removes a directory in a remote FTP server.
```ballerina
ftp:Error? rmdirResponse = ftpClient->rmdir("");
```
### FTP listener
The `ftp:Listener` is used to listen to a remote FTP location and trigger events when new files are added to or deleted from the directory. The listener supports both a generic `onFileChange` handler for file system events and format-specific content handlers (`onFileText`, `onFileJson`, `onFileXml`, `onFileCsv`, `onFile`) that automatically deserialize file content based on the file type.
An FTP listener is defined using the mandatory `protocol`, `host`, and `path` parameters. The authentication
configuration can be done using the `auth` parameter and the polling interval can be configured using the `pollingInterval` parameter.
The default polling interval is 60 seconds.
The `fileNamePattern` parameter can be used to define the type of files the FTP listener will listen to.
For instance, if the listener gets invoked for text files, the value `(.*).txt` can be given for the config.
An authentication-related configuration can be given to the FTP listener with the `auth` configuration.
##### Create a listener
The FTP Listener can be used to listen to a remote directory. It will keep listening to the specified directory and
notify on file addition and deletion periodically.
The FTP listener supports content handler methods that automatically deserialize file content based on the file type. The listener supports text, JSON, XML, CSV, and binary data types with automatic extension-based routing.
Handle text files:
```ballerina
service on remoteServer {
remote function onFileText(string content, ftp:FileInfo fileInfo) returns error? {
log:print("Text file: " + fileInfo.path);
log:print("Content: " + content);
}
}
```
Handle JSON files (as generic JSON or typed record):
```ballerina
type User record {
string name;
int age;
string email;
};
service on remoteServer {
// Handle as typed record
remote function onFileJson(User content, ftp:FileInfo fileInfo) returns error? {
log:print("User file: " + fileInfo.path);
log:print("User name: " + content.name);
}
}
```
Handle XML files (as generic XML or typed record):
```ballerina
type Config record {
string database;
int timeout;
boolean debug;
};
service on remoteServer {
// Handle as typed record
remote function onFileXml(Config content, ftp:FileInfo fileInfo) returns error? {
log:print("Config file: " + fileInfo.path);
log:print("Database: " + content.database);
}
}
```
Handle CSV files (as string arrays or typed record arrays):
```ballerina
type CsvRecord record {
string id;
string name;
string email;
};
service on remoteServer {
// Handle as array of typed records
remote function onFileCsv(CsvRecord[] content, ftp:FileInfo fileInfo) returns error? {
log:print("CSV file: " + fileInfo.path);
foreach CsvRecord record in content {
log:print("Record: " + record.id + ", " + record.name);
}
}
}
```
Handle binary files:
```ballerina
service on remoteServer {
// Handle as byte array
remote function onFile(byte[] content, ftp:FileInfo fileInfo) returns error? {
log:print("Binary file: " + fileInfo.path);
log:print("File size: " + content.length().toString());
}
}
```
Stream large files:
```ballerina
service on remoteServer {
// Stream binary content
remote function onFile(stream content, ftp:FileInfo fileInfo) returns error? {
log:print("Streaming file: " + fileInfo.path);
record {|byte[] value;|} nextBytes = check content.next();
while nextBytes is record {|byte[] value;|} {
log:print("Received chunk: " + nextBytes.value.length().toString() + " bytes");
nextBytes = content.next();
}
check content.close();
}
}
```
Stream CSV data as typed records:
```ballerina
type DataRow record {
string timestamp;
string value;
};
service on remoteServer {
// Stream CSV as records
remote function onFileCsv(stream content, ftp:FileInfo fileInfo) returns error? {
log:print("Streaming CSV file: " + fileInfo.path);
record {|DataRow value;|}|error? nextRow = content.next();
while nextRow is record {|DataRow value;|} {
log:print("Row: " + nextRow.value.timestamp + " = " + nextRow.value.value);
nextRow = content.next();
}
check content.close();
}
}
```
The FTP listener automatically routes files to the appropriate content handler based on file extension: `.txt` → `onFileText()`, `.json` → `onFileJson()`, `.xml` → `onFileXml()`, `.csv` → `onFileCsv()`, and other extensions → `onFile()` (fallback handler). You can override the default routing using the `@ftp:FunctionConfig` annotation to specify a custom file name pattern for each handler method.
### Secure access with FTPS
FTPS provides security by encrypting the control and data channels using SSL/TLS. This module supports both Explicit (starts as regular FTP and upgrades via AUTH TLS) and Implicit (SSL/TLS established immediately) modes.
##### FTPS client configuration
To use FTPS, set the protocol to FTPS and provide the secureSocket configuration including your keystore and truststore.
```ballerina
ftp:ClientConfiguration ftpsConfig = {
protocol: ftp:FTPS,
host: "ftps.example.com",
port: 21, // 21 for EXPLICIT, 990 for IMPLICIT
auth: {
credentials: { username: "user", password: "password" },
secureSocket: {
key: {
path: "/path/to/keystore.p12",
password: "keystore-password"
},
cert: {
path: "/path/to/truststore.p12",
password: "truststore-password"
},
mode: ftp:EXPLICIT, // or ftp:IMPLICIT
dataChannelProtection: ftp:PRIVATE // PROT P (Encrypted data channel)
}
}
};
ftp:Client ftpsClient = check new(ftpsConfig);
```
##### FTPS listener configuration
The listener can monitor an FTPS-enabled directory and trigger service remote functions.
```ballerina
listener ftp:Listener ftpsListener = check new({
protocol: ftp:FTPS,
host: "ftps.example.com",
port: 990,
path: "/upload",
pollingInterval: 5,
auth: {
credentials: { username: "user", password: "password" },
secureSocket: {
cert: { path: "/path/to/truststore.jks", password: "password" },
mode: ftp:IMPLICIT
}
}
});
```
##### CSV fail-safe mode
The FTP listener supports a fail-safe mode for CSV file processing When enabled, malformed CSV records are automatically skipped and written to a separate error log file in the current directory (`_error.log`), allowing the processing to continue without interruption.
This is only supported in `onFileCsv` trigger methods in ftp listener.
**Listener Configuration**:
```ballerina
listener ftp:Listener remoteServer = check new({
protocol: ftp:FTP,
host: "",
port: ,
path: "",
pollingInterval: ,
auth: {
credentials: {username: "", password: ""}
},
csvFailSafe: {
contentType: ftp:RAW_AND_METADATA
}
});
```
The `contentType` field in `csvFailSafe` specifies what information to log when a malformed CSV record is encountered.
- `METADATA` (default) - Logs only metadata about the error (location, error message)
- `RAW` - Logs the raw content of the malformed record (offending row)
- `RAW_AND_METADATA` - Logs both the raw content and metadata for comprehensive debugging
### Secure access with SFTP
SFTP is a secure protocol alternative to the FTP, which runs on top of the SSH protocol.
There are several ways to authenticate an SFTP server. One is using the username and the password.
Another way is using the client's private key. The Ballerina SFTP client and the listener support only those authentication standards.
An authentication-related configuration can be given to the SFTP client/listener with the `auth` configuration.
Password-based authentication is defined with the `credentials` configuration while the private key based authentication is defined with the `privateKey` configuration.
#### SFTP client configuration
```ballerina
ftp:ClientConfiguration sftpConfig = {
protocol: ftp:SFTP,
host: "",
port: ,
auth: {
credentials: {username: "", password: ""},
privateKey: {
path: "",
password: ""
}
}
};
```
#### SFTP listener configuration
```ballerina
listener ftp:Listener remoteServer = check new({
protocol: ftp:SFTP,
host: "",
port: ,
path: "",
pollingInterval: ,
fileNamePattern: "",
auth: {
credentials: {username: "", password: ""},
privateKey: {
path: "",
password: ""
}
}
});
```
## Issues and projects
Issues and Projects tabs are disabled for this repository as this is part of the Ballerina Standard Library. To report bugs, request new features, start new discussions, view project boards, etc. please visit Ballerina Standard Library [parent repository](https://github.com/ballerina-platform/ballerina-standard-library).
This repository only contains the source code for the library.
## Build from the source
### Set up the prerequisites
1. Download and install Java SE Development Kit (JDK) version 21 (from one of the following locations).
* [Oracle](https://www.oracle.com/java/technologies/downloads/)
* [OpenJDK](https://adoptium.net/)
> **Note:** Set the JAVA_HOME environment variable to the path name of the directory into which you installed JDK.
### Build the source
Execute the commands below to build from source.
1. To build the library:
```
./gradlew clean build
```
2. To run the tests:
```
./gradlew clean test
```
3. To run a group of tests
```
./gradlew clean test -Pgroups=
```
4. To build the library without the tests:
```
./gradlew clean build -x test
```
5. To debug library implementation:
```
./gradlew clean build -Pdebug=
```
6. To debug with Ballerina language:
```
./gradlew clean build -PbalJavaDebug=
```
7. Publish the generated artifacts to the local Ballerina central repository:
```
./gradlew clean build -PpublishToLocalCentral=true
```
8. Publish the generated artifacts to the Ballerina central repository:
```
./gradlew clean build -PpublishToCentral=true
```
## Contribute to Ballerina
As an open source project, Ballerina welcomes contributions from the community.
For more information, go to the [contribution guidelines](https://github.com/ballerina-platform/ballerina-lang/blob/master/CONTRIBUTING.md).
## Code of conduct
All contributors are encouraged to read the [Ballerina Code of Conduct](https://ballerina.io/code-of-conduct).
## Useful links
* For more information go to the [`ftp` library](https://lib.ballerina.io/ballerina/ftp/latest).
* For example demonstrations of the usage, go to [Ballerina By Examples](https://ballerina.io/learn/by-example/).
* Chat live with us via our [Discord server](https://discord.gg/ballerinalang).
* Post all technical questions on Stack Overflow with the [#ballerina](https://stackoverflow.com/questions/tagged/ballerina) tag.