pytest-elk-reporter

tl;td: https://github.com/fruch/pytest-elk-reporter

History

Few years back I’ve wrote a post about how I’ve connected python based test to ELK setup - “ELK is fun”, it was using an xunit xml, parsing it and sending it via Logstash.

Over time I’ve learn a lot about ElasticSearch and it’s friend Kibana, using them as a tool to handle logs. and also as a backend for a search component on my previous job.

So now I know logstash isn’t needed for reporting test result, posting straight into elasticsearch is easier and gives you better control, ES is doing anything “automagiclly” anyhow nowadays.

What can it do ?

• collect as much information as you can about one test results and running environment and send it to elasticsearch.
  • data like git commit/sha from current directory
  • jenkins environment variables, like job id and username triggered the job.
  • cool tracebacks which pytest excels at.
• extend - let user configure (like target address and credentials) it from code, and add data of their own to each session or test.

Tricks I’ve picked on the way

Every time I start a new package, I learn a few new things, here’s a few I’m happy about this time:

• Created the package with cookiecutter-pytest-plugin

• I’m using pre-commit

  • black - instead of the old friend autopep8.

  • pylint in both python27 and python36 to keep the code clean of silly mistakes.

• Travis

  build and test with Tox on all possible python versions.

  also uploading wheels to pypi based on tags.

  thanks to setuptools_scm version no need to edit files for setting version number.

• Appveyor

  I also cover windows own little issue (non found so far).

• Codecov

  really like that it can collect from multiple run, for example both linux and windows tests run, which helps to show the total coverage.

  got me to 93% line coverage (and going down as I add more code).

The bad stuff - a.k.a things I need to improve

• Documentation - I’ve mostly wrote test, zero docs.

• Name - it’s a bit boring, and might conflict with other plugins.

• Kibana Dashboards examples

  I should really include some sample kibana dashboard to show the strength of it, it’s own test data is a bit boring.

• auto generating and upload of docs

  I’ve tried using Travis Deploy to github Pages, didn’t exactly nailed, or got it working as I expect, maybe I should try readthedocs.

• pytest-xdist

  I still need to test it with xdist, it’s going to make thing a more tricker, as usual.

What you should next

Go grab it, git it a star, fork it.

I’m waiting for your PRs, and your stories on how it’s help you tame a whole bunch of tests.

    # forgot to show how to install :\
    pip install pytest-elk-reporter