{"id":16012765,"url":"https://github.com/smacker/bitbucket-research","last_synced_at":"2026-06-17T06:32:04.463Z","repository":{"id":141959468,"uuid":"204046810","full_name":"smacker/bitbucket-research","owner":"smacker","description":null,"archived":false,"fork":false,"pushed_at":"2019-10-04T11:22:58.000Z","size":63,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-05T00:34:44.712Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/smacker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-23T18:01:52.000Z","updated_at":"2019-08-28T11:32:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"5d261d92-9703-483b-bd05-40288ce9581f","html_url":"https://github.com/smacker/bitbucket-research","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/smacker/bitbucket-research","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smacker%2Fbitbucket-research","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smacker%2Fbitbucket-research/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smacker%2Fbitbucket-research/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smacker%2Fbitbucket-research/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/smacker","download_url":"https://codeload.github.com/smacker/bitbucket-research/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smacker%2Fbitbucket-research/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34437449,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-08T14:21:00.111Z","updated_at":"2026-06-17T06:32:04.444Z","avatar_url":"https://github.com/smacker.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Bitbucket metadata downloader research\n\nAtlassian has 3 products with the name Bitbucket.\n\n1. Bitbucket Cloud aka https://bitbucket.org\n2. Bitbucket Server\n3. Bitbucket Data Center\n\nBitbucket Cloud and Bitbucket Server are completely different products which share only the name. Bitbucket Data Center is just a distributed version of Bitbucket Server.\n\nBecause Cloud and Server are absolutely different products they provide different API. Which means we need to implement separate downloaders for them.\n\n## Cloud\n\n- Source-code: https://github.com/smacker/bitbucket-research/blob/master/cloud/main.go\n- Swagger API defenitions: https://developer.atlassian.com/bitbucket/api/2/reference/resource/\n- Go client: https://github.com/wbrefvem/go-bitbucket\n\n### Entities\n\nObjects we are interested in:\n\n- Repositories\n- Pull Requests\n- Pull Request Comments\n- Issues\n- Issue Comments (broken, need fix in the go wrapper)\n- Issues Export (doesn't work for me, maybe because none of the repos has issues)\n- Users (broken, need fix in the go wrapper)\n\n### Schema\n\n```go\ntype Repository struct {\n\tLinks *RepositoryLinks \n\t// The repository's immutable id. This can be used as a substitute for the slug segment in URLs. Doing this guarantees your URLs will survive renaming of the repository by its owner, or even transfer of the repository to a different user.\n\tUuid string \n\t// The concatenation of the repository owner's username and the slugified name, e.g. \\\"evzijst/interruptingcow\\\". This is the same string used in Bitbucket URLs.\n\tFullName string \n\tIsPrivate bool \n\tParent *Repository \n\tScm string \n\tOwner *Account \n\tName string \n\tDescription string \n\tCreatedOn time.Time \n\tUpdatedOn time.Time \n\tSize int32 \n\tLanguage string \n\tHasIssues bool \n\tHasWiki bool \n\t// Controls the rules for forking this repository.  * **allow_forks**: unrestricted forking * **no_public_forks**: restrict forking to private forks (forks cannot   be made public later) * **no_forks**: deny all forking \n\tForkPolicy string \n\tProject *Project \n\tMainbranch *Branch\n}\n\ntype Pullrequest struct {\n\tLinks *PullrequestLinks \n\t// The pull request's unique ID. Note that pull request IDs are only unique within their associated repository.\n\tId int32 \n\t// Title of the pull request.\n\tTitle string \n\tRendered *PullrequestRendered \n\tSummary *IssueContent \n\t// The pull request's current status.\n\tState string \n\tAuthor *Account \n\tSource *PullrequestEndpoint \n\tDestination *PullrequestEndpoint \n\tMergeCommit *PullrequestMergeCommit \n\t// The number of comments for a specific pull request.\n\tCommentCount int32 \n\t// The number of open tasks for a specific pull request.\n\tTaskCount int32 \n\t// A boolean flag indicating if merging the pull request closes the source branch.\n\tCloseSourceBranch bool \n\tClosedBy *Account \n\t// Explains why a pull request was declined. This field is only applicable to pull requests in rejected state.\n\tReason string \n\t// The ISO8601 timestamp the request was created.\n\tCreatedOn time.Time \n\t// The ISO8601 timestamp the request was last updated.\n\tUpdatedOn time.Time \n\t// The list of users that were added as reviewers on this pull request when it was created. For performance reasons, the API only includes this list on a pull request's `self` URL.\n\tReviewers []Account \n\t// The list of users that are collaborating on this pull request.         Collaborators are user that:          * are added to the pull request as a reviewer (part of the reviewers           list)         * are not explicit reviewers, but have commented on the pull request         * are not explicit reviewers, but have approved the pull request          Each user is wrapped in an object that indicates the user's role and         whether they have approved the pull request. For performance reasons,         the API only returns this list when an API requests a pull request by         id.         \n\tParticipants []Participant\n}\n\ntype Participant struct {\n\tUser *User \n\tRole string \n\tApproved bool \n\t// The ISO8601 timestamp of the participant's action. For approvers, this is the time of their approval. For commenters and pull request reviewers who are not approvers, this is the time they last commented, or null if they have not commented.\n\tParticipatedOn time.Time\n}\n\ntype PullrequestComment struct {\n\tId int32 \n\tCreatedOn time.Time \n\tUpdatedOn time.Time \n\tContent *IssueContent \n\tUser *User \n\tDeleted bool \n\tParent *Comment \n\tInline *CommentInline \n\tLinks *CommentLinks \n\tPullrequest *Pullrequest\n}\n\ntype Issue struct {\n\tLinks *IssueLinks \n\tId int32 \n\tRepository *Repository \n\tTitle string \n\tReporter *User \n\tAssignee *User \n\tCreatedOn time.Time \n\tUpdatedOn time.Time \n\tEditedOn time.Time \n\tState string \n\tKind string \n\tPriority string \n\tMilestone *Milestone \n\tVersion *Version \n\tComponent *Component \n\tVotes int32 \n\tContent *IssueContent\n}\n\ntype IssueComment struct {\n\tId int32 \n\tCreatedOn time.Time \n\tUpdatedOn time.Time \n\tContent *IssueContent \n\tUser *User \n\tDeleted bool \n\tParent *Comment \n\tInline *CommentInline \n\tLinks *CommentLinks \n\tIssue *Issue\n}\n```\n\n### Performance\n\nDownloading all metadata from `Unity-Technologies` organization took 5 minutes on my machine.\n\nSome stats:\n```\n$ cat output.txt | grep 'repo {' | wc -l\n      39\n$ cat output.txt | grep 'pr {' | wc -l\n     507\n$ cat output.txt | grep 'comment {' | wc -l\n     610\n$ cat output.txt | grep 'issue {' | wc -l\n       0\n```\n\n### Notes\n\n- Getting users is broken in go-wrapper\n- I could find any open source organization which uses bitbucket issues, most probably it's not popular for enterprise as well as bitbucket has integration to JIRA.\n- Need to fix issues comments\n- Export didn't work for me (most probably because project doesn't have issues)\n- Go wrapper doesn't support changing perPage parameter\n- Documentation doesn't say anything about request limits\n- Different type of authorization are supported: OAuth2, Basic HTTP, AccessToken\n- Simplest one is Basic HTTP which works similar to Github token\n\n## Server\n\n- Source-code: https://github.com/smacker/bitbucket-research/blob/master/server/main.go\n- Swagger API defenitions: https://docs.atlassian.com/bitbucket-server/rest/6.6.0/bitbucket-rest.html\n- Go client: https://github.com/gfleury/go-bitbucket-v1\n\n### Entities\n\n- Groups\n- Users\n- Projects\n- Repositories\n- Pull Requests\n- Pull Requests Comments\n- Pull Requests Tasks?\n\n### Schema\n\n```go\ntype Project struct {\n\tKey         string `json:\"key\"`\n\tID          int    `json:\"id\"`\n\tName        string `json:\"name\"`\n\tDescription string `json:\"description\"`\n\tPublic      bool   `json:\"public\"`\n\tType        string `json:\"type\"`\n\tLinks       Links  `json:\"links\"`\n}\n\ntype Repository struct {\n\tSlug          string  `json:\"slug\"`\n\tID            int     `json:\"id\"`\n\tName          string  `json:\"name\"`\n\tScmID         string  `json:\"scmId\"`\n\tState         string  `json:\"state\"`\n\tStatusMessage string  `json:\"statusMessage\"`\n\tForkable      bool    `json:\"forkable\"`\n\tProject       Project `json:\"project\"`\n\tPublic        bool    `json:\"public\"`\n\tLinks         struct {\n\t\tClone []CloneLink `json:\"clone\"`\n\t\tSelf  []SelfLink  `json:\"self\"`\n\t} `json:\"links\"`\n}\n\ntype PullRequest struct {\n\tID           int                `json:\"id\"`\n\tVersion      int32              `json:\"version\"`\n\tTitle        string             `json:\"title\"`\n\tDescription  string             `json:\"description\"`\n\tState        string             `json:\"state\"`\n\tOpen         bool               `json:\"open\"`\n\tClosed       bool               `json:\"closed\"`\n\tCreatedDate  int64              `json:\"createdDate\"`\n\tUpdatedDate  int64              `json:\"updatedDate\"`\n\tFromRef      PullRequestRef     `json:\"fromRef\"`\n\tToRef        PullRequestRef     `json:\"toRef\"`\n\tLocked       bool               `json:\"locked\"`\n\tAuthor       UserWithMetadata   `json:\"author\"`\n\tReviewers    []UserWithMetadata `json:\"reviewers\"`\n\tParticipants []UserWithMetadata `json:\"participants\"`\n\tProperties   struct {\n\t\tMergeResult       MergeResult `json:\"mergeResult\"`\n\t\tResolvedTaskCount int         `json:\"resolvedTaskCount\"`\n\t\tOpenTaskCount     int         `json:\"openTaskCount\"`\n\t} `json:\"properties\"`\n\tLinks Links `json:\"links\"`\n}\n\ntype UserWithMetadata struct {\n\tUser     UserWithLinks `json:\"user\"`\n\tRole     string        `json:\"role\"`\n\tApproved bool          `json:\"approved\"`\n\tStatus   string        `json:\"status\"`\n}\n\ntype Comment struct {\n\tID          int\n\tText        string\n\tAuthor      bitbucketv1.User\n\tCreatedDate int64\n\tUpdatedDate int64\n\tComments    []Comment\n\t// tasks\n}\n\ntype Activity struct {\n\tID            int\n\tCreatedDate   int64\n\tUser          bitbucketv1.User\n\tAction        string\n\tCommentAction string\n\tComment       Comment\n\t//commentAnchor - for comments in code\n\t//diff - for comments in code\n}\n\ntype User struct {\n\tName        string `json:\"name\"`\n\tEmail       string `json:\"emailAddress\"`\n\tID          int    `json:\"id\"`\n\tDisplayName string `json:\"displayName\"`\n\tActive      bool   `json:\"active\"`\n\tSlug        string `json:\"slug\"`\n\tType        string `json:\"type\"`\n}\n\ntype Group struct {\n\tName string\n}\n```\n\n### Performance\n\nBitbucket server accepts big numbers as perPage param. There are no request limits and downloader can be deployed close to the server. So it should be fast.\n\n### Notes\n\n- Go-wrapper doesn't have defined types for Activity or Comments\n- Go wrapper doesn't support pagination for Activities endpoint\n- Auth is similar to cloud one: OAuth2, Basic HTTP, AccessToken, APIKey\n- Api has 2 endpoints for getting users, one works well but requires additional permissions, another doesn't support pagination\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmacker%2Fbitbucket-research","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmacker%2Fbitbucket-research","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmacker%2Fbitbucket-research/lists"}