{"id":15044431,"url":"https://github.com/elastic/enterprise-search-network-drive-connector","last_synced_at":"2025-10-19T20:31:42.312Z","repository":{"id":37862495,"uuid":"441133698","full_name":"elastic/enterprise-search-network-drive-connector","owner":"elastic","description":"Official Enterprise Search | Workplace Search - Network Drives Connector","archived":false,"fork":false,"pushed_at":"2024-09-27T09:06:27.000Z","size":167,"stargazers_count":6,"open_issues_count":0,"forks_count":3,"subscribers_count":22,"default_branch":"main","last_synced_at":"2025-01-28T20:58:34.893Z","etag":null,"topics":["connector","enterprise-search","files","networkdrive","python","python3","workplace-search"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/elastic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-12-23T09:47:57.000Z","updated_at":"2024-09-27T09:06:32.000Z","dependencies_parsed_at":"2024-02-05T13:29:48.243Z","dependency_job_id":"6d08bb56-b993-4499-9b57-5a3e38c0e6c2","html_url":"https://github.com/elastic/enterprise-search-network-drive-connector","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fenterprise-search-network-drive-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fenterprise-search-network-drive-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fenterprise-search-network-drive-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/elastic%2Fenterprise-search-network-drive-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/elastic","download_url":"https://codeload.github.com/elastic/enterprise-search-network-drive-connector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237215531,"owners_count":19273550,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["connector","enterprise-search","files","networkdrive","python","python3","workplace-search"],"created_at":"2024-09-24T20:50:34.621Z","updated_at":"2025-10-19T20:31:42.305Z","avatar_url":"https://github.com/elastic.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Workplace Search: Network drives connector package\n[Elastic Enterprise Search](https://www.elastic.co/guide/en/enterprise-search/current/index.html) | [Elastic Workplace Search](https://www.elastic.co/guide/en/workplace-search/current/index.html)\n\n\u003e [!IMPORTANT]\n\u003e _**Enterprise Search will be discontinued in 9.0.**_\n\u003e\n\u003e Starting with Elastic version 9.0, we're deprecating the standalone Enterprise Search product with its included features and functionalities (including [Workplace Search](https://www.elastic.co/guide/en/workplace-search/8.x/index.html) and [App Search](https://www.elastic.co/guide/en/app-search/8.x/index.html)). They remain supported in their current form in version 8.x and will only receive security upgrades and fixes. Workplace Search Connector Packages will continue to be supported in their current form throughout 8.x versions, according to our EOL policy: https://www.elastic.co/support/eol.\n\u003e We recommend transitioning to our actively developed [Elastic Stack](https://www.elastic.co/elastic-stack) tools for your search use cases. However, if you're still using any Enterprise Search products, we recommend using the latest stable release.\n\u003e\n\u003e Here are some useful links with more information:\n\u003e * Enterprise Search FAQ: https://www.elastic.co/resources/enterprise-search/enterprise-search-faq\n\u003e * Migrating to 9.x from Enterprise Search 8.x versions: https://www.elastic.co/guide/en/enterprise-search/current/upgrading-to-9-x.html\n___\n\nUse this _Elastic Enterprise Search network drives connector package_ to deploy and run a network drive connector on your own infrastructure. The connector package extracts and syncs data from network drives. The data is indexed into a Workplace Search content source within an Elastic deployment.\n\n⚠️ _This connector package is a **beta** feature._\nBeta features are subject to change and are not covered by the support SLA of generally available (GA) features.\n\nℹ️ _This connector package requires a compatible Elastic subscription level._\nRefer to the Elastic subscriptions pages for [Elastic Cloud](https://www.elastic.co/subscriptions/cloud) and [self-managed](https://www.elastic.co/subscriptions) deployments.\n\n**Table of contents:**\n\n- [Setup and basic usage](#setup-and-basic-usage)\n  - [Gather network drives details](#gather-network-drives-details)\n  - [Gather Elastic details](#gather-elastic-details)\n  - [Create a Workplace Search API key](#create-a-workplace-search-api-key)\n  - [Create a Workplace Search content source](#create-a-workplace-search-content-source)\n  - [Choose connector infrastructure and satisfy dependencies](#choose-connector-infrastructure-and-satisfy-dependencies)\n  - [Install the connector](#install-the-connector)\n  - [Configure the connector](#configure-the-connector)\n  - [Test the connection](#test-the-connection)\n  - [Sync data](#sync-data)\n  - [Log errors and exceptions](#log-errors-and-exceptions)\n  - [Schedule recurring syncs](#schedule-recurring-syncs)\n- [Troubleshooting](#troubleshooting)\n  - [Troubleshoot extraction](#troubleshoot-extraction)\n- [Advanced usage](#advanced-usage)\n  - [Customize extraction and syncing](#customize-extraction-and-syncing)\n  - [Use document-level permissions (DLP)](#use-document-level-permissions-dlp)\n- [Connector reference](#connector-reference)\n  - [Data extraction and syncing](#data-extraction-and-syncing)\n  - [Sync operations](#sync-operations)\n  - [Command line interface (CLI)](#command-line-interface-cli)\n  - [Configuration settings](#configuration-settings)\n  - [Enterprise Search compatibility](#enterprise-search-compatibility)\n  - [Runtime dependencies](#runtime-dependencies)\n- [Connector Limitations](#connector-limitations)\n\n## Setup and basic usage\n\nComplete the following steps to deploy and run the connector:\n\n1. [Gather network drives details](#gather-network-drives-details)\n1. [Gather Elastic details](#gather-elastic-details)\n1. [Create a Workplace Search API key](#create-a-workplace-search-api-key)\n1. [Create a Workplace Search content source](#create-a-workplace-search-content-source)\n1. [Choose connector infrastructure and satisfy dependencies](#choose-connector-infrastructure-and-satisfy-dependencies)\n1. [Install the connector](#install-the-connector)\n1. [Configure the connector](#configure-the-connector)\n1. [Test the connection](#test-the-connection)\n1. [Sync data](#sync-data)\n1. [Log errors and exceptions](#log-errors-and-exceptions)\n1. [Schedule recurring syncs](#schedule-recurring-syncs)\n\nThe steps above are relevant to all users. Some users may require additional features. These are covered in the following sections:\n\n- [Customize extraction and syncing](#customize-extraction-and-syncing)\n- [Use document-level permissions (DLP)](#use-document-level-permissions-dlp)\n\n### Gather network drives details\n\nCollect the information that is required to connect to your network drives:\n\n- The domain name where the network drive is present.\n- The network drives path the connector will crawl to fetch files. This is the name of the folder shared via SMB, followed by the path relative to that shared folder. For example: if the name of the of SMB shared folder is `Users` and the folder to be fetched is `folder1`, then the path is `Users/folder1`.\n- The username the connector will use to log in to network drives.\n- The password the connector will use to log in to network drives.\n- The server IP address where the network drive is hosted.\n- The name of the server where the network drive is found. (Note that server name may not be the same as the hostname. On Windows, you can get the machine name from the System panel.)\n- The client machine name can be a random client name of up to 15 characters.\n\nℹ️ The user credentials provided for the connector must have at least **read** permissions for the folder path provided.\n\nLater, you will [configure the connector](#configure-the-connector) with these values.\n\nℹ️ The connector uses the [pysmb](https://pysmb.readthedocs.io/en/latest/) module to connect to your Network Drives. `pysmb` supports SMB1 and SMB2.\n\nSome connector features require additional details. Review the following documentation if you plan to use these features:\n\n- [Customize extraction and syncing](#customize-extraction-and-syncing)\n- [Use document-level permissions (DLP)](#use-document-level-permissions-dlp)\n\n### Gather Elastic details\n\nFirst, ensure your Elastic deployment is [compatible](#enterprise-search-compatibility) with the network drives connector package.\n\nNext, determine the [Enterprise Search base URL](https://www.elastic.co/guide/en/enterprise-search/current/endpoints-ref.html#enterprise-search-base-url) for your Elastic deployment.\n\nLater, you will [configure the connector](#configure-the-connector) with this value.\n\nYou also need a Workplace Search API key and a Workplace Search content source ID. You will create those in the following sections.\n\nIf you plan to use document-level permissions, you will also need user identity information. See [Use document-level permissions (DLP)](#use-document-level-permissions-dlp) for details.\n\n### Create a Workplace Search API key\n\nEach network drives connector authorizes its connection to Elastic using a Workplace Search API key.\n\nCreate an API key within Kibana. See [Workplace Search API keys](https://www.elastic.co/guide/en/workplace-search/current/workplace-search-api-authentication.html#auth-token).\n\n### Create a Workplace Search content source\n\nEach network drives connector syncs data from network drives into a Workplace Search content source.\n\nCreate a content source within Kibana:\n\n1. Navigate to **Enterprise Search** → **Workplace Search** → **Sources** → **Add an organization content source** → **Custom API Source**.\n2. Name your Content Source, (e.g. Network Drives Connector).\n\nFor more details see [Elastic Documentation for creating a custom content source](https://www.elastic.co/guide/en/workplace-search/current/workplace-search-custom-api-sources.html#create-custom-source).\n\nRecord the ID of the new content source. This value is labeled *Source Identifier* within Kibana. Later, you will [configure the connector](#configure-the-connector) with this value.\n\n**Alternatively**, you can use the connector’s `bootstrap` command to create the content source. See [`bootstrap` command](#bootstrap-command).\n\n### Choose connector infrastructure and satisfy dependencies\n\nAfter you’ve prepared the two services, you are ready to connect them.\n\nProvision a Windows, MacOS, or Linux server for your network drives connectors.\n\nThe infrastructure must provide the necessary runtime dependencies. See [Runtime dependencies](#runtime-dependencies).\n\nClone or copy the contents of this repository to your infrastructure.\n\n### Install the connector\n\nAfter you’ve provisioned infrastructure and copied the package, use the provided `make` target to install the connector:\n\n```shell\nmake install_package\n```\n\nThis command runs as the current user and installs the connector and its dependencies.\nNote: By Default, the package installed supports Enterprise Search version 8.0 or above. In order to use the connector for older versions of Enterprise Search(less than version 8.0) use the ES_VERSION_V8 argument while running make install_package or make install_locally command:\n\n```shell\nmake install_package ES_VERSION_V8=no\n```\n\nℹ️ Within a Windows environment, first install `make`:\n\n```\nwinget install make\n```\n\nNext, ensure the `ees_network_drive` executable is on your `PATH`. For example, on macOS:\n\n```shell\nexport PATH=/Users/shaybanon/Library/Python/3.8/bin:$PATH\n```\n\nThe following table provides the installation location for each operating system (note Python version 3.8):\n\n| Operating system | Installation location                                        |\n| ---------------- | ------------------------------------------------------------ |\n| Linux            | `./local/bin`                                                |\n| macOS            | `/Users/\u003cuser_name\u003e/Library/Python/3.8/bin`                  |\n| Windows          | `\\Users\\\u003cuser_name\u003e\\AppData\\Roaming\\Python\\Python38\\Scripts` |\n\n### Configure the connector\n\nYou must configure the connector to provide the information necessary to communicate with each service. You can provide additional configuration to customize the connector for your needs.\n\nCreate a [YAML](https://yaml.org/) configuration file at any pathname. Later, you will include the [`-c` option](#-c-option) when running [commands](#command-line-interface-cli) to specify the pathname to this configuration file.\n\n_Alternatively, in Linux environments only_, locate the default configuration file created during installation. The file is named `network_drives_connector.yml` and is located within the `config` subdirectory where the package files were installed. See [Install the connector](#install-the-connector) for a listing of installation locations by operating system. When you use the default configuration file, you do not need to include the `-c` option when running commands.\n\nAfter you’ve located or created the configuration file, populate each of the configuration settings. Refer to the [settings reference](#configuration-settings). You must provide a value for all required settings.\n\nUse the additional settings to customize the connection and manage features such as document-level permissions. See:\n\n- [Customize extraction and syncing](#customize-extraction-and-syncing)\n- [Use document-level permissions (DLP)](#use-document-level-permissions-dlp)\n\n### Test the connection\n\nAfter you’ve configured the connector, you can test the connection between Elastic and network drives. Use the following `make` target to test the connection:\n\n```shell\nmake test_connectivity\n```\n\n### Sync data\n\nAfter you’ve confirmed the connection between the two services, you are ready to sync data from network drives to Elastic.\n\nThe following table lists the available [sync operations](#sync-operations), as well as the [commands](#command-line-interface-cli) to perform the operations.\n\n| Operation                             | Command                                         |\n| ------------------------------------- | ----------------------------------------------- |\n| [Incremental sync](#incremental-sync) | [`incremental-sync`](#incremental-sync-command) |\n| [Full sync](#full-sync)               | [`full-sync`](#full-sync-command)               |\n| [Deletion sync](#deletion-sync)       | [`deletion-sync`](#deletion-sync-command)      |\n| [Permission sync](#permission-sync)   | [`permission-sync`](#permission-sync-command)   |\n\nBegin syncing with an *incremental sync*. This operation begins [extracting and syncing content](#data-extraction-and-syncing) from network drives to Elastic. If desired, [customize extraction and syncing](#customize-extraction-and-syncing) for your use case.\n\nReview the additional sync operations to learn about the different types of syncs. Additional configuration is required to use [document-level permissions](#use-document-level-permissions-dlp).\n\nYou can use the command line interface to run sync operations on demand, but you will likely want to [schedule recurring syncs](#schedule-recurring-syncs).\n\n### Log errors and exceptions\n\nThe various [sync commands](#command-line-interface-cli) write logs to standard output and standard error.\n\nTo persist logs, redirect standard output and standard error to a file. For example:\n\n```shell\nees_network_drive -c ~/config.yml incremental-sync \u003e\u003e~/incremental-sync.log 2\u003e\u00261\n```\n\nYou can use these log files to implement your own monitoring and alerting solution.\n\nConfigure the log level using the [`log_level` setting](#log_level).\n\n### Schedule recurring syncs\n\nUse a job scheduler, such as `cron`, to run the various [sync commands](#command-line-interface-cli) as recurring syncs.\n\nThe following is an example crontab file:\n\n```crontab\n0 */2 * * * ees_network_drive -c ~/config.yml incremental-sync \u003e\u003e~/incremental-sync.log 2\u003e\u00261\n0 0 */2 * * ees_network_drive -c ~/config.yml full-sync \u003e\u003e~/full-sync.log 2\u003e\u00261\n0 * * * * ees_network_drive -c ~/config.yml deletion-sync \u003e\u003e~/deletion-sync.log 2\u003e\u00261\n*/5 * * * * ees_network_drive -c ~/config.yml permission-sync \u003e\u003e~/permission-sync.log 2\u003e\u00261\n```\n\nThis example redirects standard output and standard error to files, as explained here: [Log errors and exceptions](#log-errors-and-exceptions).\n\nUse this example to create your own crontab file. Manually add the file to your crontab using `crontab -e`. Or, if your system supports cron.d, copy or symlink the file into `/etc/cron.d/`.\n\n⚠️ **Note**: It's possible that scheduled jobs may overlap.\nTo avoid multiple crons running concurrently, you can use [flock](https://manpages.debian.org/testing/util-linux/flock.1.en.html) with cron to manage locks. The `flock` command is part of the `util-linux` package. You can install it with `yum install util-linux`\nor `sudo apt-get install -y util-linux`.\nUsing flock ensures the next scheduled cron runs only after the current one has completed execution. \n\nLet's consider an example of running incremental-sync as a cron job with flock:\n\n```crontab\n0 */2 * * * /usr/bin/flock -w 0 /var/cron.lock ees_network_drive -c ~/config.yml incremental-sync \u003e\u003e~/incremental-sync.log 2\u003e\u00261\n```\n\n## Troubleshooting\n\nTo troubleshoot an issue, first view your [logged errors and exceptions](#log-errors-and-exceptions).\n\nUse the following section to help troubleshoot further:\n\n- [Troubleshoot extraction](#troubleshoot-extraction)\n\nIf you need assistance, use the Elastic community forums or Elastic support:\n\n- [Enterprise Search community forums](https://discuss.elastic.co/c/enterprise-search/84)\n- [Elastic Support](https://support.elastic.co)\n\n### Troubleshoot extraction\n\nThe following sections provide solutions for content extraction issues.\n\n#### Issues extracting content from attachments\n\nThe connector uses the [Tika module](https://pypi.org/project/tika/) for parsing file contents from attachments. [Tika-python](https://github.com/chrismattmann/tika-python) uses Apache Tika REST server. To use this library, you need to have Java 7+ installed on your system as tika-python starts up the Tika REST server in the background.\n\nAt times, the TIKA server fails to start hence content extraction from attachments may fail. To avoid this, make sure Tika is running in the background.\n\n#### Issues extracting content from images\n\nTika Server also detects contents from images by automatically calling Tesseract OCR. To allow Tika to also extract content from images, you need to make sure tesseract is on your path and then restart tika-server in the background (if it is already running). For example, on a Unix-like system, try:\n\n```shell\nps aux | grep tika | grep server # find PID\nkill -9 \u003cPID\u003e\n```\n\nTo allow Tika to extract content from images, you need to manually install Tesseract OCR.\n\n## Advanced usage\n\nThe following sections cover additional features that are not covered by the basic usage described above.\n\nAfter you’ve set up your first connection, you may want to further customize that connection or scale to multiple connections.\n\n- [Customize extraction and syncing](#customize-extraction-and-syncing)\n- [Use document-level permissions (DLP)](#use-document-level-permissions-dlp)\n\n## Customize extraction and syncing\n\nThe connector will support the following rules for filtering files:\n\n* Size: To filter all files of size **\u003e** x bytes, add `\u003ex` to the `include/exclude` field.\n* Path Template: To index files in specific subdirectories or files with specific names/types, use a glob pattern. \n\nThe Path Template can be used for following use cases:\n\n1. Folder: For example `/Engineering/**/*` will filter all the content inside the Engineering folder. \n1. File Name: To filter files containing a given string pattern, use the following glob pattern:\n\n- `**/\u003cfilename\u003e.*` : files named `\u003cfilename\u003e`.\n- `**/main*.*` : filenames that start with the substring `main`.\n        \n1. File Type: To filter `pdf` media type files, use the glob pattern `**/*.pdf`.\n\nThese can be defined under include/exclude parameters.\n\nFinally, you can set custom timestamps to control which objects are synced, based on their created or modified timestamps. [Configure](#configure-the-connector) the following settings:\n\n- [`start_time`](#start_time)\n- [`end_time`](#end_time)\n\n### Use document-level permissions (DLP)\n\nComplete the following steps to use document-level permissions:\n\n1. Enable document-level permissions\n1. Map user identities\n1. Sync document-level permissions data\n\n#### Enable document-level permissions\n\nWithin your configuration, enable document-level permissions using the following setting: [`enable_document_permission`](#enable_document_permission).\n\n#### Map user identities\n\nCopy to your server a CSV file that provides the mapping of user identities. The file must follow this format:\n\n- First column: network drive user's/group's SID\n- Second column: Elastic username\n\nThen, configure the location of the CSV file using the following setting: [`network_drives_enterprise_search_user_mapping`](#network_drives_enterprise_search_user_mapping).\n\n#### Sync document-level permissions data\n\nSync document-level permissions data from network drives to Elastic.\n\nThe following sync operations include permissions data:\n\n- [Permission sync](#permission-sync)\n- [Incremental sync](#incremental-sync)\n\nSync this information continually to ensure correct permissions. See [Schedule recurring syncs](#schedule-recurring-syncs).\n\n## Connector reference\n\nThe following reference sections provide technical details:\n\n- [Data extraction and syncing](#data-extraction-and-syncing)\n- [Sync operations](#sync-operations)\n- [Command line interface (CLI)](#command-line-interface-cli)\n- [Configuration settings](#configuration-settings)\n- [Enterprise Search compatibility](#enterprise-search-compatibility)\n- [Runtime dependencies](#runtime-dependencies)\n\n### Data extraction and syncing\n\nEach network drives connector extracts and syncs files from network drives:\n\nIt extracts content from various document formats, and it performs optical character recognition (OCR) to extract content from images.\n\nYou can customize extraction and syncing per connector. See [Customize extraction and syncing](#customize-extraction-and-syncing).\n\n### Sync operations\n\nThe following sections describe the various operations to [sync data](#sync-data) from network drives to Elastic.\n\n#### Incremental sync\n\nSyncs to Enterprise Search all files *created or modified* since the previous incremental sync.\n\nWhen [using document-level permissions (DLP)](#use-document-level-permissions-dlp), each incremental sync will also perform a [permission sync](#permission-sync).\n\nPerform this operation with the [`incremental-sync` command](#incremental-sync-command).\n\n#### Full sync\n\nSyncs to Enterprise Search all files *created or modified* since the configured [`start_time`](#start_time). Continues until the current time or the configured [`end_time`](#end_time).\n\nPerform this operation with the [`full-sync` command](#full-sync-command).\n\n#### Deletion sync\n\nDeletes from Enterprise Search all files *deleted* since the previous deletion sync.\n\nPerform this operation with the [`deletion-sync` command](#deletion-sync-command).\n\n#### Permission sync\n\nSyncs to Enterprise Search all file document permissions since the previous permission sync.\n\nWhen [using document-level permissions (DLP)](#use-document-level-permissions-dlp), use this operation to sync all updates to users and groups within network drives.\n\nPerform this operation with the [`permission-sync` command](#permission-sync-command).\n\n### Command line interface (CLI)\n\nEach network drives connector has the following command line interface (CLI):\n\n```shell\nees_network_drive [-c \u003cpathname\u003e] \u003ccommand\u003e\n```\n\n#### `-c` option\n\nThe pathname of the [configuration file](#configure-the-connector) to use for the given command.\n\n```shell\nees_network_drive -c ~/config.yml full-sync\n```\n\n#### `bootstrap` command\n\nCreates a Workplace Search content source with the given name. Outputs its ID.\n\n```shell\nees_network_drive bootstrap --name 'Accounting documents' --user 'shay.banon'\n```\n\nSee also [Create a Workplace Search content source](#create-a-workplace-search-content-source).\n\nTo use this command, you must [configure](#configure-the-connector) the following settings:\n\n- [`enterprise_search.host_url`](#enterprise_searchhost_url-required)\n- [`workplace_search.api_key`](#workplace_searchapi_key-required)\n\nAnd you must provide on the command line any of the following arguments that are required:\n\n- `--name` (required): The name of the Workplace Search content source to create.\n- `--user` (optional): The username of the Elastic user who will own the content source. If provided, the connector will prompt for a password. If omitted, the connector will use the configured API key to create the content source.\n\n#### `incremental-sync` command\n\nPerforms a [incremental sync](#incremental-sync) operation.\n\n#### `full-sync` command\n\nPerforms a [full sync](#full-sync) operation.\n\n#### `deletion-sync` command\n\nPerforms a [deletion sync](#deletion-sync) operation.\n\n#### `permission-sync` command\n\nPerforms a [permission sync](#permission-sync) operation.\n\n### Configuration settings\n\n[Configure](#configure-the-connector) any of the following settings for a connector:\n\n#### `network_drive.domain` (required)\n\nThe domain name where your network drives is present.\n\n```yaml\nnetwork_drive.domain: \"Domain3\"\n```\n\n#### `network_drive.username` (required)\n\nThe username of the account for network drives. See [Gather network drives details](#gather-network-drives-details).\n\n```yaml\nnetwork_drive.username: bill.gates\n```\n\n#### `network_drive.password` (required)\n\nThe password of the account to be used for crawling the network drives. See [Gather network drives details](#gather-network-drives-details).\n\n```yaml\nnetwork_drive.password: 'L,Ct%ddUvNTE5zk;GsDk^2w)(;,!aJ|Ip!?Oi'\n```\n\n#### `network_drive.path` (required)\n\nThe network drives path the connector will crawl to fetch files.\n\n```yaml\nnetwork_drive.path: 'path1/drives/org1/ws'\n```\n\n#### `network_drive.server_name` (required)\n\nThe name of the server where the network drive is found. (Note that server name may not be the same as the hostname. On Windows, you can get the machine name from the System panel.)\n\n```yaml\nnetwork_drive.server_name: 'elastic'\n```\n\n#### `network_drive.server_ip` (required)\n\nThe server ip where network drives is hosted.\n\n```yaml\nnetwork_drive.server_ip: '1.2.3.4'\n```\n\n#### `client_machine.name` (required)\n\nThe client machine name can be a random client name of up to 15 characters.\n\n```yaml\nclient_machine.name: 'Workplace'\n```\n\n#### `enterprise_search.api_key` (required)\n\nThe Workplace Search API key. See [Create a Workplace Search API key](#create-a-workplace-search-api-key).\n\n```yaml\nenterprise_search.api_key: 'zvksftxrudcitxa7ris4328b'\n```\n\n#### `enterprise_search.source_id` (required)\n\nThe ID of the Workplace Search content source. See [Create a Workplace Search content source](#create-a-workplace-search-content-source).\n\n```yaml\nenterprise_search.source_id: '12345678909876543ab21012a'\n```\n\n#### `enterprise_search.host_url` (required)\n\nThe [Enterprise Search base URL](https://www.elastic.co/guide/en/enterprise-search/current/endpoints-ref.html#enterprise-search-base-url) for your Elastic deployment.\nNote: To use Enterprise Search version 8 or above, port is required.\n\n```yaml\nenterprise_search.host_url: https://my-deployment.ent.europe-west1.gcp.cloud.es.io:9243\n```\n\n#### `enable_document_permission`\n\nWhether the connector should sync [document-level permissions (DLP)](#use-document-level-permissions-dlp) from network drives.\n\nBy default, it is set to `Yes` i.e. the connector will try to sync document-level permissions.\n\n```yaml\nenable_document_permission: Yes\n```\n\n#### `include/exclude`\n\nSpecifies which files should be indexed based on their size or path template in network drives.\n\nNote: By default, the connector includes files listed in the include field and excludes files listed in the exclude field.\n\n```yaml\ninclude:\n   size:\n   path_template: [\"**/*.txt\", \"**/*.contact\", \"**/*.docx\", \"**/*.json\", \"**/*.png\", \"**/*.jpg\", \"**/*.jpeg\", \"**/*.py\", \"**/*.yml\", \"**/*.md\", \"**/*.ini\", \"**/*.sh\", \"**/*.rst\", \"**/*.pdf\", \"**/*.rtf\", \"**/*.ppt\", \"**/*.file\"]\nexclude:\n   size: [\"\u003e10000000\"]\n   path_template:\n```\n\n#### `start_time`\n\nA UTC timestamp the connector uses to determine which objects to extract and sync from network drives. Determines the *starting* point for a [full sync](#full-sync).\nSupports the following time format `YYYY-MM-DDTHH:MM:SSZ`\n\n```yaml\nstart_time: 2022-04-01T04:44:16Z\n```\n\nBy default, it is set to unix epoch time `1970-01-01T00:00:00Z`\n\n#### `end_time`\n\nA UTC timestamp the connector uses to determine which objects to extract and sync from network drives. Determines the *stopping* point for a [full sync](#full-sync).\nIt supports the following time format YYYY-MM-DDTHH:MM:SSZ\n\n```yaml\nend_time: 2022-04-01T04:44:16Z\n```\n\nBy default, it is set to current execution time.\n\n#### `log_level`\n\nThe level or severity that determines the threshold for [logging](#log-errors-and-exceptions) a message. One of the following values:\n\n- `DEBUG`\n- `INFO` (default)\n- `WARN`\n- `ERROR`\n\n```yaml\nlog_level: INFO\n```\n\nBy default, it is set to `INFO`.\n\n#### `retry_count`\n\nThe number of retries to perform when there is a server error. The connector applies an exponential backoff algorithm to retries.\n\n```yaml\nretry_count: 3\n```\n\nBy default, it is set to `3`.\n\n#### `network_drives_sync_thread_count`\n\nThe number of threads the connector will run in parallel when fetching documents from the network drive. By default, the connector uses 5 threads.\n\n```yaml\nnetwork_drives_sync_thread_count: 5\n```\n\n#### `enterprise_search_sync_thread_count`\n\nThe number of threads the connector will run in parallel when indexing documents into the Enterprise Search instance. By default, the connector uses 5 threads.\n\n```yaml\nenterprise_search_sync_thread_count: 5\n```\n\nFor the Linux distribution with atleast 2 GB RAM and 4 vCPUs, you can increase the thread counts if the overall CPU and RAM are under utilized i.e. below 60-70%.\n\n#### `network_drives_enterprise_search_user_mapping`\n\nThe pathname of the CSV file containing the user identity mappings for [document-level permissions (DLP)](#use-document-level-permissions-dlp).\n\n```yaml\nnetwork_drives_enterprise_search_user_mapping: 'C:/Users/banon/connector/identity_mappings.csv'\n```\n\n#### Enterprise Search compatibility\n\nThe network drives connector package is compatible with Elastic deployments that meet the following criteria:\n\n- Elastic Enterprise Search version greater than or equal to 7.13.0.\n- An Elastic subscription that supports this feature. Refer to the Elastic subscriptions pages for [Elastic Cloud](https://www.elastic.co/subscriptions/cloud) and [self-managed](https://www.elastic.co/subscriptions) deployments.\n\n#### Runtime dependencies\n\nEach network drives connector requires a runtime environment that satisfies the following dependencies:\n\n- Windows, MacOS, or Linux server. The connector has been tested with CentOS 7, MacOS Monterey v12.0.1, and Windows 10.\n- Python version 3.6 or later.\n- To extract content from images: Java version 7 or later, and [`tesseract` command](https://github.com/tesseract-ocr/tesseract) installed and added to `PATH`\n- To schedule recurring syncs: a job scheduler, such as `cron`\n\n## Connector Limitations\n\n:information_source: Currently, the connector will only work for Windows and Linux shared SMB folders.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Fenterprise-search-network-drive-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Felastic%2Fenterprise-search-network-drive-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Felastic%2Fenterprise-search-network-drive-connector/lists"}