{"id":19573437,"url":"https://github.com/ess-dmsc/vmm-sdat","last_synced_at":"2025-04-27T05:32:37.486Z","repository":{"id":52169076,"uuid":"162710569","full_name":"ess-dmsc/vmm-sdat","owner":"ess-dmsc","description":"VMM3a/SRS Data Analysis Tool: Analysis software for VMM3a data, recorded with the SRS as PCAP or HDF5 files (GdGEM pipeline of the EFU)","archived":false,"fork":false,"pushed_at":"2024-10-23T14:12:25.000Z","size":81854,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-23T16:28:53.900Z","etag":null,"topics":["hdf5-files","pcap-files","root-cern"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ess-dmsc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-12-21T12:16:16.000Z","updated_at":"2024-10-23T14:12:29.000Z","dependencies_parsed_at":"2024-10-23T00:01:46.852Z","dependency_job_id":"0fd65315-ff1f-44be-90fb-9ffd276cb3b0","html_url":"https://github.com/ess-dmsc/vmm-sdat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ess-dmsc%2Fvmm-sdat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ess-dmsc%2Fvmm-sdat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ess-dmsc%2Fvmm-sdat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ess-dmsc%2Fvmm-sdat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ess-dmsc","download_url":"https://codeload.github.com/ess-dmsc/vmm-sdat/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224061455,"owners_count":17249243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hdf5-files","pcap-files","root-cern"],"created_at":"2024-11-11T06:33:46.453Z","updated_at":"2024-11-11T06:33:47.822Z","avatar_url":"https://github.com/ess-dmsc.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# vmm-sdat\n\nVMM3a/SRS Data Analysis Tool: Analysis software for VMM3a data, recorded with the SRS or the ESS readout as PCAPNG file. From the PCAP file, a root tree with the hits and clusters is created.\nFor more information about ROOT see [here](https://root.cern.ch/)\n\n## Getting Started\n\n### Which branch do I need?\nThe main branch analyses PCAPNG files in the SRS or ESS data format. PCAPNG files can be generated with Wireshark or tcdump. The SRS branch analyses PCAPNG files, or HDF5 files that have been created with the GdGEM pipeline of the ESS DAQ/event formation unit. The old SRS data format (offsets 0-31) and the new SRS data format (offsets -1 to 15) are supported.\n\n### Prerequisites\n- Boost system library [find_package( Boost REQUIRED COMPONENTS system)]\n- root6 from https://root.cern.ch/downloading-root [find_package(ROOT REQUIRED)]\n- Installation of ESS DAQ https://github.com/ess-dmsc/essdaq or event formation unit https://github.com/ess-dmsc/event-formation-unit\n\n\n## Installing\n\nTo build the program, start in the program directory vmm-sdat\n```\nmkdir build\ncd build\ncmake .. or (to enable debug info) cmake -DENABLE_DTRACE=ON ..\n```\nThe program uses conan to install all the ESS dependencies.\n\n## Built With\n* [CMAKE](https://cmake.org/) - Cross platform makefile generation\n\n## Deployment\nHow to run the program:\nThe executable is called convertFile, it has been created in the build directory. For help concerning the possible parameters, type:\n```\n./convertFile \n```\n\nExample command line:\n```\n./convertFile -f data.pcapng \n-vmm \"[[1,0,2,2],[1,0,2,3],[1,0,2,0],[1,0,2,1],[1,1,2,8],[1,1,2,9],[1,1,2,6],[1,1,2,7]]\" \n-axis \"[[1,0],0],[[1,1],0]\" -bc 40 -tac 60 -th 0 -cs 1 -ccs 2 -dt 200 -mst 1 -spc 500 \n-dp 200 -coin center-of-mass -crl 0.5 -cru 2 -save [[1],[1],[1]] -json 0 -n 0 -algo 0 -swap 0 \n-cal CalibrationFile.json -df SRS\n\n```\n\n## Description of analysis program\n\nThe ESS DAQ https://github.com/ess-dmsc/essdaq provides in various detector pipelines the option to write hits \nto hdf5 files. The format of this hdf5 file is defined in the ESS DAQ in the Readout.h files. For GEM detectors \nand the SRS readout, the Readout.h file can be found in \nhttps://github.com/ess-dmsc/event-formation-unit/blob/master/src/gdgem/nmx/Readout.h\n\n### Time calculation\nThe convertFile utility of the vmm-sdat package analyses pcapng directly from the SRS FEC, or the hdf5 files of the \ngdgem/SRS pipeline. There are two time stamps in the hdf5 file, the srs_timestamp coming from the SRS front end card (FEC), and the \nchiptime coming from the VMM ASIC. To obtain the total time, these two timestamps are added, and the new field in the \nroot file is just called time.\n\nThe VMM measures time in BCID and TDC, with a 40 MHz BC clock the BCID has a 25 ns resolution. The BCID is a 12 bit \nnumber with values going from 0-4095, that means the bc_time is covering a range of 4096 * 25 ns = 102.4 us. \nThe TDC gives information about the time between BCIDs. With a TAC slope of \n60 ns, the time resolution of the tdc_time is 60ns/256 bits = 0.23 ns. The tdc_time and the bc_time together are \ncalled chiptime. \n\nThe formula is chiptime = bc_time - tdc_time = (BCID + 1) * 25 ns - TDC * 60ns / 256. \nThe TDC time is subtracted, because the TDC starts counting when the peakfinder has found the hit, and is then \nstopped by the falling edge of the next BC clock pulse. A small TDC value means the hit occured shortly before \nthe next BC clock, whereas a large TDC value means it appeared a long time before the next clock. \n\nEvery 4096 * 25 ns the BCID overflows, these overflows are called offset in the FEC firmware. \nExample:\n    Offset 24, BCID 100, TDC 80: 24 * 102.4 us + (100+1) * 25 ns - 80 * 60 ns/256 = 2460106.25 ns = 2.46 ms\nBut even with the offset the time range covered is only 32 * 102.4 us = 3278.8 us = 3.3 ms. Therefore every \n3.3 ms 42 bit time markers are send by the FEC card. All the offsets always refer to the previous time marker \nsent for the particular VMM. The EFU now takes the time markers and the offset and calculates the srs_timestamp \nfrom it in unit [ns]. If one analyses a pcapng file, the vmm-sdat analysis code carries out the same steps that are \ndone in the EFU. The chiptime is calculated from BCID and TDC, the srs_timestamp is calculated from the last time marker\nand the offsets.\n\n### Calibration files\nThe ESS DAQ has the option to load JSON calibration files, that contain for each channel of each VMM ASIC \nan ADC (adc_offset and adc_slope) and a time (adc_offset and adc_slope) correction.\nhttps://github.com/ess-dmsc/event-formation-unit/blob/master/src/gdgem/srs/CalibrationFile.cpp\nThe JSON calibration file can either be produced automatically with the VMM Slow Control program\nhttps://gitlab.cern.ch/rd51-slow-control/vmmsc\nor by other means. If the calibration file is loaded during the data acquisition with the ESS DAQ, then the \nchip_time has been calculated using the following formula:\n- new_chiptime = BCID * 25 ns + ( 1.5*25 ns - tdc_time - time_offset) * time_slope\nFor the ADC, the formula is: \n- new_adc = (adc - adc_offset) * adc_slope;\nIf the data has been saved to file without the use of calibration files in the ESS DAQ, then the calibration\ncan be loaded during the analysis by convertFile using the parameter -cal.\n\n### Hits\nAs first step, the hits are stored in vectors (optionally also in a branch of the root tree), depending on the \nmapping of the VMM ASICs to the detectors and detector planes. Then, to create clusters for each detector plane, \nthe hits are first sorted in time (which is the sum of srs_time and chiptime). \n\n### Time clusters\nA time cluster contains all hits with subsequent timestamps, that have a time difference smaller or equal than \nthe value defined in the -dt (delta t) parameter. Further, the time difference between the first and the last hit \nhas to be smaller than the time span defined in the -spc (span cluster) parameter. \n\n### Plane clusters\nThe time clusters are then sorted by strip to create the plane clusters. For hits to belong to one cluster, the gap \nbetween neighbouring strips cannot be larger than the -mst (missing strips) parameter. For each of the plane clusters,\na position and a time is calculated using three different algorithms:\n    - center-of-mass (charge as weight) \n    - center-of-mass2 (charge squared as weight)\n    - utpc (latest time)\nThe user can define additional algorithms in the method Clusterer::AlgorithmUTPC() in\nhttps://github.com/ess-dmsc/vmm-sdat/blob/master/src/Clusterer.cpp\n\nThe additional algorithm is picked depending on the -algo parameter. At the moment, two additional algorithms are \ndefined there \n    - 0 = utpc center-of-mass2 (center of mass squared of the strip with the latest time and its one or two neighbours)\n    - 1 = utpc center-of-mass (center of mass of the strip with the latest time and its one or two neighbours)\nIf the number of strips in a plane cluster is equal or larger than the value specified in the -cs (cluster size) \nparameter, the cluster is valid. For X-ray data the cluster size parameter is usually 1 or 2, whereas for electron or\nalpha tracks it can be 3 or larger. \nValid plane clusters are optionally stored in a branch of the root tree. For pad detectors the clusters appear with the \nx-indices of the pads in plane 0 of the detector, and with the y-indices in plane 1 of the detector.  \n\n### Detector clusters\nTo determine now the detector clusters, the plane clusters in the two detector planes are matched depending on their\ncluster times. The time difference between the cluster times of two plane clusters has to be smaller or equal than the \nvalue specified in the -dp (delta planes) value. The user has to choose in the -coin (coincidence) parameter, which of \nthe four cluster times calculated with the different algorithms has to be used for the matching. The sum of all hists \nin the two planes of the detector cluster has to be larger or equal to the -ccs (coincident cluster size) parameter.\nIn a detector with equal charge sharing between the two planes, the charge of a detector cluster should be almost \nidentical in the two detector planes. The -ratio (charge ratio) parameter defines the allowed charge ratio between\nthe two planes (charge plane 0 / charge plane 1) or (charge plane 1 / charge plane 0). For pad detectors, the detector\nclusters contain the x-index in the pos0 entry, and the y-index in the pos1 entry. Detector that are only 1D, contain the \nplane clusters also in the detector cluster tree. The entries of the missing plane (either plane 0 or plane 1) are set to zero.\n\n## Explanation of parameters\n  \n    -f: data file with the extension .pcapng. The data file was created by Wireshark or tcdump\n    \n    -df: data format of pcapng file. Default is SRS format (valid offsets: -1 - 15), other option is data format ESS \n    \n    Definition of detector geometry: EITHER the flags -vmm, -axis and -mapping can be used, OR a JSON geometry file loaded with -geo.\n\n    -vmm: mapping of detectors, plane, fecs and chips starting and ending with \" and separated by brackets\n        and comma [[det, plane, fec,chip], [det, plane, fec, chip], etc.]\n        The tuples for the VMMs are defined as follows:\n            detector (choose a number between 0 and 255)\n            plane (0 or 1)\n            fec (fecID set in firmware based on IP address, 10.0.0.1 is fecID 1, 10.0.0.2 is fecID 2 and so on)\n            vmm (depends on connection of hybrid to FEC, FEC channel 1 equals VMMs 0 and 1, FEC channel 2 \n                VMMs 2 and 3, FEC channel 8 VMMs 14 and 15)\n        When looking at the detector, the following conventions are used:\n            - top side of the hybrids is visible (if the hybrids are mounted in the readout plane)\n            - side of the Hirose connector (bottom of the hybird) is visible (hybrids mounted on detector side)\n            - plane 0 is at the bottom (HDMI cables go downwards)\n            - plane 1 is at the right side (HDMI cables go to the right)\n        If one looks at a VMM3a hybrid (connector to detector readout is on the bottom side), \n            the channel 0 of the VMM 0 is always where the HDMI cable is connected\n        If the planes are correctly used as described above, the VMM IDs are always in icreasing order \n            PER HYBRID (e.g. 14, 15 or e.g. 0, 1)\n\n    -axis: direction of axis. Detector, plane and direction flag (if direction flag = 1, axis direction is flipped). \n   \tFor a 1D detector,  only axis 0 is defined, for a 2D detector axis 0 and 1 have to be defined.\n        Detector, plane and direction flag starting and ending with \" and separated by bracket and comma \n            [[[det,plane],flag], [[det, plane],flag]]\n        The tuples for the axes are defined as follows:\n            - detector (choose a number between 0 and 255)\n            - plane (0 or 1)\n            - flip axis flag (0 or 1)\n        Using the convention described above, if the plane axis is NOT FLIPPED:\n            - plane 0 is at the bottom and goes from left (0) to right (255)\n            - plane 1 is at the right and goes from bottom (0) to top (255)\n        If the plane axis is FLIPPED:\n            - plane 0 is at the bottom and goes from right (0) to left (255)\n            - plane 1 is at the right and goes from top (0) to bottom (255)\n            \n    -map:   Mapping of VMM3a channels to strips of the detector readout. There are some pre-defined options:\n    \t- gem: channels are continuously mapped to strips, channel 0 is strip 0, channel 127 strip 127 (default).\n         - gem_swapped: odd and even channels are swapped (to correct error in readout).\n         - mm1: Micromegas mapping from Jona Bortfeld\\n\"  \u003c\u003c std::endl;\n    \n    -geo:   Instead of using -vmm, -axis, -map, the detector geometry can be defined in a JSON file. Tow example geometry files (for strip and pad detectors) are in the run folder.\n\n    -sc: Scale coordinates. Per detector a tuple with three values in mm, \n        e.g for two detectors [[s0,s1,s2], [s0,s1,s2]]\n    \n    -tl: Translate coordinates. Per detector a tuple with three values in mm, \n        e.g for two detectors [[t0,t1,t2], [t0,t1,t2]]\n    \n    -ro: Rotate around plane 0, plane 1, plane 2. Per detector a tuple with three angles in degrees, \n        e.g for two detectors [[r0,r1,r2], [r0,r1,r2]]\n    \n    -tr: Transform detector coordinates. \n        S=scale, T=translate, R0=rotation plane 0, R1=rotation plane1, R2=rotation plane2\n        example (two detectors): -tr [[S,T,R2], [S,T,R0]]. \n        First detector scaling, then translation, then rotation around normal axis to plane0 and plane 1.\n        Second detector scaling, then translation, then rotation around plane0 axis\n        Normally beam in direction of plane 2 or z, hence: Plane 0 (x), plane 1 (y), plane 2 (z)\n   \n    -bc: bunch crossing clock. Optional argument (default 40 MHz)\n\n    -tac: tac slope. Optional argument (default 60 ns)\n\n    -th: threshold value in ADC counts. Optional argument (default 0, if -1, only hits with over threshold flag 1 are accepted)\n\n    -cs: minimum cluster size per plane. Optional argument (default 1)\n\n    -ccs: minimum cluster size in plane 0 and plane 1 together. Optional argument (default 2)\n\n    -dt: maximum time difference between strips in time sorted vector. Optional argument (default 200)\n\n    -mst: maximum missing strips in strip sorted vector. Optional argument (default 2)\n\n    -spc: maximum time span of cluster in one dimension (determined by drift size and speed). \n        Optional argument (default 500)\n\n    -dp: maximum time between matched clusters in x and y. Optional argument (default 200)\n\n    -coin: Valid clusters normally occur at the same time in plane 0 and plane 1 of a detctor. \n        The parameter -dp determines the permitted time difference between the planes\n        The time can be calculated with the center-of-mass algorithm (center-of-mass), the uTPC method (utpc) \n        or the center-of-mass squared method (charge2). Optional argument (default center-of-mass)\n\n    -algo:  There are three different algorithms implemented that calulate cluster times: \n        center-of-mass (charge as weight), \n        center-of-mass2 (charge squared as weight),\n        utpc (latest time)\n        An additional algorithm can be chosen in pos_algo and time_algo field in clusters\n        0: utpc with COG\n        1: utpc with COG2\n        2: COG including only over Threshold hits\n        3: COG2 including only over Threshold hits\n        4: position and time of largest ADC\n        5: trigger pattern (NIP box), the trigger pattern is stored as integer in time_algo2\n            The vmm that is connected to the NIP box has to be defined as plane 2 of the detector. The channels of the VMM have to be mapped to the strips in the form channel representing bit 0 = strip 0, bit 1 channel = strip 1 and so on.\n\n    -crl: Valid clusters normally have the same amount of charge in both detector planes \n    \t(ratio of charge plane 0 / charge plane 1 is 100% or 1). Depending on the readout, \n    \tthe charge sharing can be different, e.g. in a standard GEM strip readout the total \n    \tcharge is divided 60/40 between plane 0/plane 1.\n\t\tWith -crl one sets the lower threshold for the plane0/plane1 charge ratio. \n\t\tOptional argument (default 0.5).\n\n\t-cru: With -cru one sets the upper threshold for the plane0/plane1 charge ratio. \n\t\tOptional argument (default 2).\n\n\t-swap: Same connectors on readout boards unintentionally swap odd and even channels. \n\t\tWith -swap 1 one can correct this. Optional parameter (default 0).\n\n    -save:  select which data to store in root file. Input is a list of lists of detectors, e.g. [[1,2],[1,2],[1,2,3]]\n            first list : detectors for which to write the hits (hit is a VMM3a channel over threshold)\n            second list : clusters plane\n            third list : clusters detector\n            Examples:\n                [[1,2],[],[]]: hits for detectors 1 and 2 only \n                [[],[],[1,2]]: clusters detector for detector 1 and 2 only \n                [[2],[1],[1]]: hits for detector 2, clusters plane, clusters detector for detector 1 \n   \n    -stats: Show statistics of the run (default 1, show all stats)\n\n    -json: create a json file of the detector images. Optional argument (default 0)\n\n    -n: number of hits to analyze. Optional argument (default 0, i.e. all hits)\n\n    -cal: Name of the calibration file. A calibration file is a JSON file containing an \n    \tADC and/or time correction in the form of a slope and an offset correction. \n    \tThe calibration file can be produced with the VMM slow control tool. Optional parameter. \n\n    -t0: SRS data format: Time correction in ns for each vmm in the format\n        [[fec0, vmm0,correction0], [fec1, vmm1, correction1]]. The correction value is subtracted from all timestamps. \n        If instead of a number the word 'run' is put as correction, the first timestamp of the run is used as correction. For the ESS data format, the first timestamp is always substracted to get smaller timestamps.\n        Optional argument for SRS data format (default 0)\n\n## Running the program\nIn the run folder there is a script and example data that show how to use the convertFile utility. \nrunConversion.py converts the Wireshark pcapng file example.pcapng into a root tree.\nBuilding on this script you can construct your own scripts for file conversion. The meaning of the \ncommand line parameters is explained above. For sure you will have to change the mapping of FECs and VMMs\nto the axes of your detector.\n  \n\n## Accessing the produced ROOT tree\nThe script accessTree in the build directory shows how to access the produced root trees. The data in the \nroot trees can be further analyzed and plotted. An example.root tree is provided in the source directory.\nThe time of the first pcapng packet is stored in the file as date and as epochtime in seconds. You can retrieve the values like that:\nTString t  = f-\u003eGet(\"unixtime\")-\u003eGetTitle()\nTString d  = f-\u003eGet(\"date\")-\u003eGetTitle()\n\n## Authors\n\n* **Dorothea Pfeiffer** - *Initial work, implementation of new features* - [dorotheapfeiffer](https://github.com/dorotheapfeiffer)\n* **Lucian Scharenberg** - *Debugging, testing, proposing new features, Python analysis*  - [lscharenberg](https://github.com/lscharenberg)\n\nSee also the list of [contributors](https://github.com/ess-dmsc/vmm-sdat/contributors) who participated in this project.\n\n## License\n\nThis project is licensed under the BSD 2-Clause \"Simplified\" License - see the [license](https://github.com/ess-dmsc/vmm-sdat/contributors/LICENSE.md) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fess-dmsc%2Fvmm-sdat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fess-dmsc%2Fvmm-sdat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fess-dmsc%2Fvmm-sdat/lists"}