{"id":19448556,"url":"https://github.com/neurodata/ndparse","last_synced_at":"2025-04-25T02:31:26.133Z","repository":{"id":73813738,"uuid":"51901962","full_name":"neurodata/ndparse","owner":"neurodata","description":"NeuroData Parse","archived":false,"fork":false,"pushed_at":"2019-07-15T18:23:06.000Z","size":7670,"stargazers_count":3,"open_issues_count":3,"forks_count":2,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-03T15:04:56.606Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/neurodata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-02-17T07:00:26.000Z","updated_at":"2019-07-15T18:23:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"76a46c48-105b-4bae-ac29-24bcf89472b0","html_url":"https://github.com/neurodata/ndparse","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurodata%2Fndparse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurodata%2Fndparse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurodata%2Fndparse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neurodata%2Fndparse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/neurodata","download_url":"https://codeload.github.com/neurodata/ndparse/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250741940,"owners_count":21479703,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T16:27:35.857Z","updated_at":"2025-04-25T02:31:26.121Z","avatar_url":"https://github.com/neurodata.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n**ndparse** is the NeuroData Parse parent repository\n\nndparse:  NeuroData Parse\n=========================\n![Downloads shield](https://img.shields.io/pypi/dm/ndparse.svg)\n\nInstallation\n------------\n\nYou can either clone this repository and use it locally, or install from pypi:\n\n~~~bash\npip install ndparse\n~~~\n\nUse this Python library to easily interface with NeuroData algorithms to\nmanually annotate data, use computer vision algorithms and deploy\nacross large neuroscience datasets.\n\nThis repo contains the code needed to train, evaluate, and deploy code for parsing volumes of NeuroData images. \n\nIt contains the legacy code for manno and macho.  The current version of ndod is divided into the several major components required to parse neuroscience data at scale:\n\n- **annotate**: manual annotation protocols and tools\n- **algorithms**: big-data research algorithms to inform neuroscience\n- **learn**: protocols for training and validation\n- **deploy**: machine annotation for deployment and scaling.\n- **utils**:  Utilities and convenience functions (e.g., plotting) for big data neuroscience\n\nTo just use one of these, say **mana**, in python you can (and should) type the following: `from ndparse import mana`\n\n*This repo is under extremely active development during the first quarter of 2016.  The previous version of mano, macho and ndod code may be found in [ndod](https://github.com/neurodata/ndod).  The core code that is used for computer vision by the neurodata team will be transitioned to a pip installable python package in the next few weeks.  Stay tuned.*\n\nOn osx:\nvigra 1.10.0 is needed (1.11.x currently causes a segfault)\nblosc 1.2.3 is needed (1.3.x doesn't compile)\n\nThis is accounted for in the scripts below and doesn't affect performance.\n~~~bash\npip install conda\nconda create -n ndp -c ilastik ilastik-everything-but-tracking -y\nsource activate ndp\nconda uninstall vigra -y\nconda install --channel https://conda.anaconda.org/FlyEM vigra -y\nconda install ipython notebook -y\nconda install requests gcc blosc -y\npip install scikit-learn --upgrade\npip install scikit-image --upgrade\npip install mahotas\npip install blosc==1.2.0\npip install ndio\npip install ndparse\n~~~\n\n\nFor ilastik processing:\n\n~~~python\n\nfrom time import time\nt = time()\nimport ndparse as ndp\nimport numpy as np\nimport ndio.remote.neurodata as neurodata\nnd = neurodata()\npad = 20\ninput_data = nd.get_cutout('bock11','image',22000-pad,23000+pad,22000-pad,23000+pad,3000-pad,3020+pad,resolution=1)\nclassifier = 'bock11_v0.ilp' #specify this!\nprobs = ndp.algorithms.run_ilastik_pixel(input_data, classifier,threads=4, ram=4000)\nprobs = ndp.utils.choose_channel_4d_3d(probs, 1)\n\n\nblank = np.where(sum(sum(input_data\u003e0)) == 0)\nprobs[:,:,blank] = 0\nprobs = np.float16(probs)\nnp.save('probs_temp.npy',probs)\nprint 'time elapsed: ' + str(time()-t)\nndp.plot(probs,slice=2)\n\n~~~\n# To convert a probability cube to objects using ndparse, one could use the following sequence:\n\n# 3D probs array\n\nimport numpy as np\nimport ndparse as ndp\nprobs = np.load('probs_temp.npy')\nobj = ndp.algorithms.basic_objectify(probs,threshold=0.5, min_size=10,max_size=5000)\nndp.plot(obj,slice=10)\n\n~~~\n\nTo plot ndio obtained (RAMON or numpy array) data:\n\n~~~python\nimport ndparse as ndp\nimport ndio.remote.neurodata as neurodata\nnd = neurodata()\ntoken = 'kasthuri11cc'\nchannel = 'image'\nim2 = nd.get_volume('ac3ac4','ac4_synapse_truth', 4400,5400, 5440, 6440, 1100, 1102, resolution=1)\nim = nd.get_volume(token, channel, 4400, 5400, 5440, 6440, 1100, 1102, resolution=1)\nndp.plot(im,im2,slice=1, alpha=0.5)\n~~~\n\n\nffmpeg library:\nhttps://github.com/imageio/imageio-binaries/tree/master/ffmpeg\n\n\nimport ndio.remote.neurodata as neurodata\nnd = neurodata()\nimport ndio.ramon as ramon\nimport numpy as np\nr = ramon.RAMONSegment()\nc = np.round(np.random.rand(5,5)*100,0)\nc = np.uint8(c)\nr.cutout = c\nnd.post_ramon('ndio_demos','ramontests',r)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneurodata%2Fndparse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneurodata%2Fndparse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneurodata%2Fndparse/lists"}