{"id":25180972,"url":"https://github.com/vmagamedov/metricslog","last_synced_at":"2026-05-16T11:31:47.351Z","repository":{"id":57441225,"uuid":"75630471","full_name":"vmagamedov/metricslog","owner":"vmagamedov","description":"Essentials for application metrics and structured logging","archived":false,"fork":false,"pushed_at":"2018-05-02T14:14:49.000Z","size":37,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-08T06:53:29.176Z","etag":null,"topics":["elasticsearch","kibana","logging","logs","logstash","metrics","rsyslog","syslog"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vmagamedov.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-12-05T14:05:20.000Z","updated_at":"2018-05-02T14:14:50.000Z","dependencies_parsed_at":"2022-09-02T06:53:01.018Z","dependency_job_id":null,"html_url":"https://github.com/vmagamedov/metricslog","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/vmagamedov/metricslog","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fmetricslog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fmetricslog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fmetricslog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fmetricslog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vmagamedov","download_url":"https://codeload.github.com/vmagamedov/metricslog/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmagamedov%2Fmetricslog/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33100813,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T04:41:52.686Z","status":"ssl_error","status_checked_at":"2026-05-16T04:41:52.009Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","kibana","logging","logs","logstash","metrics","rsyslog","syslog"],"created_at":"2025-02-09T16:38:15.371Z","updated_at":"2026-05-16T11:31:47.325Z","avatar_url":"https://github.com/vmagamedov.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Metrics collection and submission as structured logs. And all you need to setup\nstructured logging for regular application logs.\n\nExamples\n~~~~~~~~\n\n- ``examples/logstash`` – send structured logs directly into `Logstash`_ via UDP\n  or TCP protocol for further processing using dozens of `Logstash`_ plugins,\n  among with *elasticsearch* output plugin for storing logs in `Elasticsearch`_\n  and analysing them using `Kibana`_;\n- ``examples/syslog`` – send structured logs using `rsyslog`_ and `CEE`_ format\n  for fast and reliable logs collection and processing. With `rsyslog`_ you will\n  also have ability to store logs in `Elasticsearch`_ and analyse them using\n  `Kibana`_ by using *omelasticsearch* output module;\n- ``examples/console`` – stream colored structured logs into *stderr*, Python\n  tracebacks are also highlighted using `Pygments`_ library (if installed).\n\nIssues with sending logs into Elasticsearch\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nField types in Elasticsearch are static. Fields can be created dynamically, but\nit's type can't be changed afterward. This means that once you indexed\n``{\"foo\": 1}`` document into Elasticsearch, you wouldn't be able to index document\n``{\"foo\": \"bar\"}``, because ``foo`` field was created with ``long`` type, which\nis incompatible with ``string`` type.\n\nIn order to overcome this issue, you can use sophisticated mapping template for\nElasticsearch:\n\n- ``examples/es2-template.yaml`` - mapping template for Elasticsearch 2.x\n- ``examples/es5-template.yaml`` - mapping template for Elasticsearch 5.x\n\nThese templates are designed to treat all unknown fields as ``keyword`` type,\nwhich can accept strings and numbers, and this field can be used for\nfiltering purposes. This field is actually a multi-field, with ``key``, ``num``\nand ``time`` subfields, which can be used for aggregation and sorting purposes,\nif the indexed value is looking like numbers or timestamps.\n\nSo, for example, when we indexed ``{\"foo\": \"123\"}`` document, we will be able to\nsearch this fields with ``foo:123`` query and we will be able to aggregate this\nfield as numeric field using ``foo.num`` subfield.\n\n**Note**: there is one more limitation in Elasticsearch, which can't be fixed by\nsuch mapping template. Elasticseach has explicit ``object`` type, along with other\nscalar types. And they all share the same namespace in the fields mapping. This\nmeans, that if you indexed object into field ``foo``, you wouldn't be able to\nindex any other data type into this field, only objects. So be very careful with\nwhat you're indexing into Elasticsearch, use objects only as namespaces with\nunique and self-describing names and **prefer flat structures with scalar types**\ninstead of deeply nested objects.\n\nInstallation\n~~~~~~~~~~~~\n\n.. code-block:: shell\n\n    pip install metricslog\n\nExample\n~~~~~~~\n\nRegular structured logging for application logs are possible using ``extra``\nkeyword argument:\n\n.. code-block:: python\n\n    import logging\n\n    log = logging.getLogger('some.app.module.name')\n\n    log.info('As you can see, %s and %s logs', 'structured', 'formatted',\n             extra={'user': 123})\n\nLogged message will be looking like this in the console:\n\n.. code-block:: shell\n\n    2017-01-01 00:00:00,000 INFO some.app.module.name As you can see, structured and formatted logs user=123\n\nand like this when sent into syslog (formatted for readability):\n\n.. code-block:: javascript\n\n    \u003c14\u003eapp-name: @cee: {\"@timestamp\": \"2015-10-26T09:00:00.000Z\",\n                         \"@version\": \"1\",\n                         \"message\": \"As you can see, structured and formatted logs\",\n                         \"logger\": \"some.app.module.name\",\n                         \"user\": 123}\n\n.. _Logstash: https://www.elastic.co/products/logstash\n.. _Elasticsearch: https://www.elastic.co/products/elasticsearch\n.. _Kibana: https://www.elastic.co/products/kibana\n.. _CEE: http://cee.mitre.org\n.. _rsyslog: http://www.rsyslog.com\n.. _Pygments: http://pygments.org\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmagamedov%2Fmetricslog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvmagamedov%2Fmetricslog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmagamedov%2Fmetricslog/lists"}