{"id":25895437,"url":"https://github.com/diamondlightsource/fast_dp","last_synced_at":"2025-08-03T13:14:30.773Z","repository":{"id":46084344,"uuid":"83571637","full_name":"DiamondLightSource/fast_dp","owner":"DiamondLightSource","description":"Fast DP: Fast Data Processsing with XDS","archived":false,"fork":false,"pushed_at":"2025-04-07T20:45:16.000Z","size":1342,"stargazers_count":7,"open_issues_count":8,"forks_count":12,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-07-04T02:07:50.496Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pypi.python.org/pypi/fast_dp","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DiamondLightSource.png","metadata":{"files":{"readme":"README.rst","changelog":"HISTORY.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-01T15:45:02.000Z","updated_at":"2025-01-16T16:52:36.000Z","dependencies_parsed_at":"2023-09-27T18:24:41.979Z","dependency_job_id":"32279f16-c2de-428f-803e-4cae37212de2","html_url":"https://github.com/DiamondLightSource/fast_dp","commit_stats":{"total_commits":220,"total_committers":9,"mean_commits":"24.444444444444443","dds":0.5454545454545454,"last_synced_commit":"4ea4775955826230f92e4246c6cccc4df87ef8b7"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"purl":"pkg:github/DiamondLightSource/fast_dp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Ffast_dp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Ffast_dp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Ffast_dp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Ffast_dp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DiamondLightSource","download_url":"https://codeload.github.com/DiamondLightSource/fast_dp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DiamondLightSource%2Ffast_dp/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268550815,"owners_count":24268447,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-03T02:00:12.545Z","response_time":2577,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-02T22:31:22.189Z","updated_at":"2025-08-03T13:14:30.750Z","avatar_url":"https://github.com/DiamondLightSource.png","language":"Python","readme":"=======================================\nFast DP: Fast Data Processsing with XDS\n=======================================\n\n.. image:: https://img.shields.io/pypi/v/fast_dp.svg\n        :target: https://pypi.python.org/pypi/fast_dp\n        :alt: PyPI release\n\n.. image:: https://img.shields.io/pypi/pyversions/fast_dp.svg\n        :target: https://pypi.org/project/fast-dp/\n        :alt: Supported Python versions\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n        :target: https://github.com/ambv/black\n        :alt: Code style: black\n\nIntroduction\n------------\n\nFast DP is a small Python program which uses XDS, CCP4 \u0026 CCTBX to deliver\ndata processing results very quickly: quite how quickly will depend on the\noperating environment. In essence, the first image in the sweep is passed\nto the program, its header read and then XDS used to index with a triclinic\nlattice using spots drawn from small wedges of data around the start, 45\ndegrees in and 90 degrees in (or as close as possible to this). Integration\nis then performed in parallel, either using multiple cores or multiple\nprocessors if the XDS forkintegrate script is appropriately configured. The\ndata are then scaled with XDS, still in P1, before analysis with Pointless.\nFinally the analysis from Pointless and the global postrefinement results\nfrom the XDS CORRECT step are then used to select a pointgroup, after which\nthe data are re-scaled with XDS in this pointgroup and merged with Aimless.\n\nAt Diamond Light Source, using an appropriately configured cluster with a\nparallel file store, this process typically takes up to two minutes for any\nnumber of images.\n\nUsage\n^^^^^\n\n::\n\n  fast_dp -h\n  Usage: fast_dp.py [options]\n\n  Options:\n    -h, --help            show this help message and exit\n    -b BEAM, --beam=BEAM  Beam centre: x, y (mm)\n    -a ATOM, --atom=ATOM  Atom type (e.g. Se)\n    -j NUMBER_OF_JOBS, --number-of-jobs=NUMBER_OF_JOBS\n                          Number of jobs for integration\n    -k NUMBER_OF_CORES, --number-of-cores=NUMBER_OF_CORES\n                          Number of cores for integration\n    -J MAXIMUM_NUMBER_OF_JOBS, --maximum-number-of-jobs=MAXIMUM_NUMBER_OF_JOBS\n                          Maximum number of jobs for integration\n    -c CELL, --cell=CELL  Cell constants for processing, needs spacegroup\n    -s SPACEGROUP, --spacegroup=SPACEGROUP\n                          Spacegroup for scaling and merging\n    -1 FIRST_IMAGE, --first-image=FIRST_IMAGE\n                          First image for processing\n    -N LAST_IMAGE, --last-image=LAST_IMAGE\n                          Last image for processing\n    -r RESOLUTION_HIGH, --resolution-high=RESOLUTION_HIGH\n                          High resolution limit\n    -R RESOLUTION_LOW, --resolution-low=RESOLUTION_LOW\n                          Low resolution limit\n\nConventional usage, e.g. on laptop, would be e.g:\n\n::\n\n  fast_dp ~/data/i04-BAG-training/th_8_2_0001.cbf\n\ngiving the following output on a 2011 Macbook Pro:\n\n::\n\n  Fast_DP installed in: /Users/graeme/svn/fast_dp\n  Starting image: /Users/graeme/data/i04-BAG-training/th_8_2_0001.cbf\n  Number of jobs: 1\n  Number of cores: 0\n  Processing images: 1 -\u003e 540\n  Phi range: 82.00 -\u003e 163.00\n  Template: th_8_2_####.cbf\n  Wavelength: 0.97625\n  Working in: /private/tmp/fdp\n  All autoindexing results:\n  Lattice      a      b      c  alpha   beta  gamma\n       tP  57.80  57.80 150.00  90.00  90.00  90.00\n       oC  81.80  81.70 150.00  90.00  90.00  90.00\n       oP  57.80  57.80 150.00  90.00  90.00  90.00\n       mC  81.80  81.70 150.00  90.00  90.00  90.00\n       mP  57.80  57.80 150.00  90.00  90.00  90.00\n       aP  57.80  57.80 150.00  90.00  90.00  90.00\n  Mosaic spread: 0.04 \u003c 0.06 \u003c 0.07\n  Happy with sg# 89\n   57.80  57.80 150.00  90.00  90.00  90.00\n  --------------------------------------------------------------------------------\n        Low resolution  28.89  28.89   1.37\n       High resolution   1.34   5.99   1.34\n                Rmerge  0.062  0.024  0.420\n               I/sigma  13.40  44.70   1.60\n          Completeness   99.6   98.9   96.1\n          Multiplicity    5.3    5.0    2.8\n    Anom. Completeness   96.5  100.0   71.4\n    Anom. Multiplicity    2.6    3.1    1.2\n     Anom. Correlation   99.9   99.9   76.0\n                 Nrefl 306284   3922  11217\n               Nunique  57886    786   4030\n             Mid-slope  1.007\n                  dF/F  0.075\n            dI/sig(dI)  0.823\n  --------------------------------------------------------------------------------\n  Merging point group: P 4 2 2\n  Unit cell:  57.78  57.78 150.01  90.00  90.00  90.00\n  Processing took 00h 03m 59s (239 s) [306284 reflections]\n  RPS: 1277.6\n\nThe main result is the file fast_dp.mtz containing the scaled and merged\nintensities, a log file from Aimless for plotting the merging statistics\nand the information above in fast_dp.log.\n\nSee also fast_rdp to rerun last steps to change choices.\n\nIf you find fast_dp useful please cite |fastdp_doi_badge| as a DOI for the\nsource code and / or:\n\n    `Winter, G. \u0026 McAuley, K. E.\n    \"Automated data collection for macromolecular crystallography.\"\n    Methods 55, 81-93 (2011).\n    \u003chttps://doi.org/10.1016/j.ymeth.2011.06.010\u003e`_\n\nPlease also cite XDS, CCTBX \u0026 CCP4:\n\n    `Kabsch, W.\n    \"XDS.\"\n    Acta Cryst. D66, 125-132 (2010)\n    \u003chttps://doi.org/10.1107/S0907444909047337\u003e`_\n\n    `Grosse-Kunstleve, R. W., Sauter, N. K., Moriarty, N. W., and Adams, P. D.\n    \"The Computational Crystallography Toolbox: crystallographic algorithms\n    in a reusable software framework\"\n    J. Appl. Cryst. (2002). 35, 126-136\n    \u003chttps://doi.org/10.1107/S0021889801017824\u003e`_\n\n    `Winn, M. D. et al.\n    \"Overview of the CCP4 suite and current developments\"\n    Acta. Cryst. D67, 235-242 (2011)\n    \u003chttps://doi.org/10.1107/S0907444910045749\u003e`_\n\nDependencies\n------------\n\nfast_dp depends on:\n\n* XDS\n* CCP4\n* CCTBX\n\nIf all of these are installed and configured no further work is needed. For\nparallel operation in integration a forkintegrate script is needed to send\njobs to your queuing system.\n\nInstallation\n------------\n\nYou can install the latest release version of fast_dp from PyPI by loading\nyour CCTBX environment and then running\n\n::\n\n  libtbx.pip install fast_dp\n\nand update an existing installation to a newer version with\n\n::\n\n  libtbx.pip install --upgrade fast_dp\n\nYou will then have to run eg.\n\n::\n\n  libtbx.configure libtbx\n\nto make sure all command line programs are set up correctly.\n\nInstallation for developers\n^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf you are a developer then you can run\n\n::\n\n  libtbx.install fast_dp\n\ninstead. This will check out a development copy of fast_dp into the cctbx\nmodules directory and then install that to the system. To update your\ndevelopment copy you will need to update the repository as usual and then\nrun\n\n::\n\n  libtbx.python setup.py develop\n\nin the source directory.\n\nCoding Standards\n^^^^^^^^^^^^^^^^\n\nWith prejudice the style guide for fast_dp is consistent PEP8 as\nimplemented by black https://black.readthedocs.io/en/stable/ -\ninstallation is close to trivial (pip3 install black) and run *with no\noptions* i.e. in fast_dp directory\n\n::\n\n  black .\n\nwill do what is needed to return the formatting to the defaults so\nthat the diffs show only the code diffs not any formatting\ndifferences. There is no intention to be heavy handed about this, but\nhaving a style guide helps developers who contribute as there is no doubt.\n\nAssumptions\n-----------\n\nThe XDS.INP files generated by fast_dp make the following assumptions:\n\n* All scans are about a single axis, approximately parallel to the detector\n  \"fast\" axis (multi-axis goniometers are fine provided the axis for the\n  scan is fixed)\n* The detector is not offset in two-theta i.e. the beam is approximately\n  perpendicular to the detector face.\n* Currently templates are included for Pilatus 2M \u0026 6M, ADSC and Rayonix CCD\n  detectors - modification to other detectors may be possible.\n\nSupport\n-------\n\nfast_dp is provided with no guarantee of support however \"best effort\" support\nwill be provided on contacting scientificsoftware@diamond.ac.uk. Users may be\nasked to provide example data in the event of a bug report.\n\nAcknowledgements\n----------------\n\nfast_dp was developed at Diamond Light Source with the specific purpose of\nproviding feedback to users about the merging statistics of their data in the\nshortest possible time. Clearly, however, it is very much dependent on XDS\nand its intrinsic parallelisation as well as CCP4 and CCTBX to operate, and\nwithout these fast_dp could not exist.\n\nLicense\n-------\n\nCopyright 2014 Diamond Light Source\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nRelease Process\n---------------\n\nMake sure you have written up your changes in the\n`HISTORY.rst \u003chttps://github.com/DiamondLightSource/fast_dp/blob/master/HISTORY.rst\u003e`_\nfile.\n\nTo prepare a new fast_dp release you need to install\n`bump2version \u003chttps://pypi.org/project/bump2version/\u003e`_,\nfor example by running\n\n::\n\n  pip install bump2version\n\nor using ``libtbx.pip`` in an CCTBX environment, followed by a\n``libtbx.configure``. Releases can then be made by:\n\n::\n\n  # Assuming current version is 1.1.1\n  bumpversion major  # 1.1.1 -\u003e 2.0.0\n      # or\n  bumpversion minor  # 1.1.1 -\u003e 1.2.0\n      # or\n  bumpversion patch  # 1.1.1 -\u003e 1.1.2\n\n  git push\n  git push origin v1.1.2\n      # or\n  git push origin v1.2.0\n      # or\n  git push origin v2.0.0\n\nThe release tag, once pushed to Github, will be picked up by Travis\nwhich will generate a new package and upload it directly to PyPI.\n\n\n.. |fastdp_doi_badge| image:: https://zenodo.org/badge/DOI/10.5281/zenodo.13039.svg\n      :align: top\n      :target: https://doi.org/10.5281/zenodo.13039\n      :alt: 10.5281/zenodo.13039\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiamondlightsource%2Ffast_dp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdiamondlightsource%2Ffast_dp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdiamondlightsource%2Ffast_dp/lists"}