{"id":20100597,"url":"https://github.com/seriyps/xhttpc","last_synced_at":"2025-09-20T13:31:45.874Z","repository":{"id":9502906,"uuid":"11396534","full_name":"seriyps/xhttpc","owner":"seriyps","description":"Extensible HTTP Client for  Erlang","archived":false,"fork":false,"pushed_at":"2014-07-17T15:59:40.000Z","size":214,"stargazers_count":25,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-03T16:23:41.657Z","etag":null,"topics":["erlang","http-client","middleware"],"latest_commit_sha":null,"homepage":null,"language":"Erlang","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"apache/maven-release","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seriyps.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-07-14T01:07:40.000Z","updated_at":"2021-12-07T16:27:19.000Z","dependencies_parsed_at":"2022-09-08T05:04:35.429Z","dependency_job_id":null,"html_url":"https://github.com/seriyps/xhttpc","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriyps%2Fxhttpc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriyps%2Fxhttpc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriyps%2Fxhttpc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seriyps%2Fxhttpc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seriyps","download_url":"https://codeload.github.com/seriyps/xhttpc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233665637,"owners_count":18710919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["erlang","http-client","middleware"],"created_at":"2024-11-13T17:16:27.278Z","updated_at":"2025-09-20T13:31:40.616Z","avatar_url":"https://github.com/seriyps.png","language":"Erlang","funding_links":[],"categories":[],"sub_categories":[],"readme":"xhttpc - eXtensible HTTP(S) Client for Erlang\n=============================================\n\nEasily extensible - just add your own middlewares.\nSimple tiny core - all the additional functionality splitted to middleware modules.\nNo extra processes and message passing.\nNot a http client, but http client wrapper - built on top of lhttpc HTTP client,\nsupport for other HTTP clients (httpc, hackney, ibrowse etc) can be added easily.\nBatteries included - see below.\n\nInspired mostly by python's [urllib2](http://docs.python.org/2/library/urllib2.html).\nBattle-tested in production system.\n\nAPI\n---\n\n```erlang\n%% Start underlying HTTP client library\n% if you use OTP's httpc\ninets:start().\n% if you use lhttpc (recommended)\n[application:start(App) || App \u003c- [crypto, public_key, ssl, lhttpc]].\n\n%% Initialize session (passing list of middleware names or pairs {middleware_name, init_args})\nSession = xhttpc:init(\n    [compression_middleware,\n     cookie_middleware,\n     {defaults_middleware, [{append_hdrs, [{\"User-Agent\", \"xhttpc 0.0.1\"}]},\n                            {default_options, [{timeout, 10}]}]}]),\n\n%% Perform HTTP request, using positional arguments\n{S2, {ok, {{200, StatusLine}, Headers, Body}}} =\n    xhttpc:request(Session, \"http://example.com/\",\n                   post, [{\"Content-Type\", \"application/x-www-form-urlencoded\"}],\n                   \u003c\u003c\"a=b\"\u003e\u003e, [{timeout, 10000}]),\n\n%% Perform HTTP request, using `#request{}` record API\n-include_lib(\"xhttpc/include/xhttpc.hrl\").\n{S3, {ok, {{200, StatusLine}, Headers, Body}}} =\n    xhttpc:request(S2, #request{url=\"http://example.com/\"}),\n\n%% Perform HTTP request, using proplists API\n{S4, {ok, {{200, StatusLine}, Headers, Body}}} =\n    xhttpc:request(S3,\n                   \"http://example.com/\",\n                   [{method, post},\n                    {body, \"a=b\"},\n                    {headers, [{\"Content-Type\", \"application/x-www-form-urlencoded\"}]}]),\n\n%% Interact with middleware's state\n{S5, Cookies} = xhttpc:call(S4, cookie_middleware, {get_cookies, \"example.com\", \"/\"}).\n\n%% Terminate session\nok = xhttpc:terminate(S5).\n```\n\nApi sequence diagram:\n```\n API call                      | compression_mw |     | defaults_mw   |     | lhttpc/httpc/etc |\n-------------------------------+----------------+-----+---------------+-----+------------------+\nxhttpc:request/2           --\u003e |  request/3     | --\u003e |  request/3    | --\u003e | lhttpc:request/5 |\n                               |                |     |               |     |       VVV        |\nresponse()                 \u003c-- |  response/4    | \u003c-- |  response/4   | \u003c-- |     response()   |\n\nxhttpc:call(defaults_mw, Arg)\u003c-:----------------:---\u003e |  call/2       |     |                  |\n\nxhttpc:(init|terminate)    --\u003e | init|terminate |     |               |     |                  |\n                           \\---:----------------:---\u003e | init|terminate|     |                  |\n```\n\nTry `rebar doc` to generate HTML API documentation from sources.\n\nFAQ\n---\n\nQ: I think API is too verbose / I don't like records in API.\n\nA: Just make your own wrappers / shortcuts.\n\n\nQ: I don't like this session-chains `S1, S2, S3...`.\n\nA: You can store sessions in a process / gen_server / public ETS or any storage\nyou like, eg:\n\n```erlang\n% this will spawn new gen_server and store session in it.\nPid = my_wrapper:init([...]).\n% following calls will lookup / update session from storage using `Pid` as identifier.\n{ok, {{200, StatusLine}, Headers, Body}} = my_wrapper:get(Pid, \"http://example.com/\").\nResult = my_wrapper:call(Pid, ...).\nok = my_wrapper:terminate(Pid).\n```\nIf you'll create shared storage for sessions (like public ETS table), you can also use\nyour worker proces's `self()` pid as session ID and don't pass `Pid` parameter at all.\n\nBatteries included\n------------------\n\n* Traffic compression (compression_middleware)\n* Redirects, with loop detection (redirect_middleware)\n* Cookies (cookie_middleware + RFC6265 cookie parser / serializer)\n* Automatic http referer (referer_middleware)\n* Constant default / additional headers and options for each request (defaults_middleware)\n* TODO: basic HTTP auth middleware\n\nTODO\n----\n\n* Support for partial upload\n  actually supported, but just need to create some wrappers\n* Support for partial download\n  eg, for compression middleware\n* More examples:\n  ETS cookie storage\n  API with external session storage (to avoid S1, S2, S3, SN problem)\n* More http client  adapters:\n  * DONE httpc\n  * hackney\n  * ibrowse\n\nHow to make your own middleware\n-------------------------------\n\nAs an example, let's implement middleware, named 'logging_middleware'. It will log\nsome user-defined uniq session ID, request method, url, response code and run-time to error_logger.\nMiddleware must implement `xhttpc_middleware` behaviour (see xhttpc_middleware.erl).\n\nThis include following methods:\n\n* `init/1` - will be called when user initialize session by call `xhttpc:init/1`.\n* `request/3` - before request will be sended to server.\n* `response/4` - before reply will be returned to user.\n* `terminate/2` - when user terminates session by call `xhttpc:terminate/2`, and\n* `call/2` - if user wish to call concrete middleware like `xhttpc:call(logging_middleware, Args)`.\n\nSo, first we add standard module header:\n\n```erlang\n-module(logging_middleware).\n-behaviour(xhttpc_middleware).  % declare behaviour\n\n-export([init/1, request/3, response/4, terminate/2, call/2]).\n-include_lib(\"xhttpc/include/xhttpc.hrl\").  % #request{} definition there.\n\n-record(state, {session_id, timer_start, n_requests=0}).  % middleware's internal state\n```\n\nNow, let's implement some callback functions:\n\n```erlang\ninit([SessionID]) -\u003e\n  {ok, #state{session_id=SessionID}}.\n```\nSo, when user create new session `S = xhttpc:init([{logging_middleware, [\"MySessionID\"]}]).`, this\nfunction will be called and it return `{ok, #state{session_id=\"MySessionID\"}}`.\nWe return #state{} just like gen_server's state - it will be passed to each\nAPI function call as last argument later.\n\n```erlang\nrequest(Session, Request, #state{n_requests=NReqs} = State) -\u003e\n    Begin = os:timestamp(),\n    {update, Session, Request, State#state{timer_start=Begin,\n                                           n_requests=NReqs + 1}}.\n```\nNow, when user call `xhttpc:request/2,6`, request first goes to each middlewares `request/3`\ncallback. We only capture current timestamp before request been executed\n(sended to the server) and increment requests counter.\n\n```erlang\nresponse(Session, #xhttpc_request{url=Url, method=Method}, Response,\n         #state{timer_start=Start, n_requests=NReqs, session_id=SID} = State) -\u003e\n    Runtime = timer:now_diff(os:timestamp(), Start) / 1000000,\n    Status = case Response of\n        {ok, {{Code, _}, _Hdrs, _Body}} -\u003e\n            Code;\n        {error, Reason} -\u003e\n            Reason\n    end,\n    % lager:info(...)\n    error_logger:info_msg(\"Request #~p of session ~p; Method: ~p Url: ~s runtime: ~p status: ~p\",\n                          [NReqs, SID, Method, Url, Runtime, Status]),\n    noaction.\n```\nAfter request has been executed and response been returned by underlying HTTP client,\nmiddleware's `response/4` will be called (in reverse order). We calculate request's runtime as diff\nof saved in `State` request-start-time and current timestamp. Next, if response\nwas successful - extract HTTP response code, if not - error reason. And finally,\nlog all the data to `error_logger`. Note, that we return atom `noaction` as a flag\nthat Session, State and Response hasn't been modified by this callback.\n\n```erlang\nterminate(Reason, #state{session_id=SID}) -\u003e\n    error_logger:info_msg(\"Session ~p has been terminated with reason ~p\", [SID, Reason]),\n    ok.\n```\n\nNo comments. Just callback for `xhttpc:terminate/1,2`.\n\nNow imagine, that we need to get number of currently executed requests with that\nsession. That's why we have `call/2` method here. First, we may implement\nuser-friendly API method for it:\n\n```erlang\n-export([get_requests_count/1]).\n\nget_requests_count(Session) -\u003e\n    xhttpc:call(Session, ?MODULE, get_requests_count).\n```\nThis is just a wrapper around `xhttpc:call/3`. Next, we implement a callback\n\n```erlang\ncall(get_requests_count, #state{n_requests=NReqs} = State) -\u003e\n    {NReqs, State}.\n```\n\nSo simple! Isn't it? Note that we return 2-tuple from `call/2` - that way we can\nnot only retrieve some data from middleware's state, but can also mutate it's state\n(for example, we can change it's `session_id` or reset requests counter on the fly).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseriyps%2Fxhttpc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseriyps%2Fxhttpc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseriyps%2Fxhttpc/lists"}