Grafana with Influxdb and python client

Recently I came across this tool called Grafana. Its a tool that you can use to show graphs and statistics of all kinds. The most popular combination for setup , is Grafana with influxdb as a database. 

I am sharing my setup steps, and supporting python scripts that can be used to generate information and write back to the database. And finally we get to see some nice graphs.

Note: I am using a Mac with Catalina OS 10.15. Python 3.6+.

Installation Plan:

1. Influxdb

2. Grafana

3. influxdb client module for Python.

• influxdb_client (for influxdb 1.8 or 2.x)
• influxdb (for 1.1.1 or less)
  

Docker : Most useful commands

Docker has a lot to talk about. But I am going to list a few commands here that come handy while working with it on a daily basis. We will list them in a sequence that looks like a tutorial.

1. List all docker containers

docker container ls

2. List all images

docker image ls

docker images

3. Spawn & run from an image, a new container.

docker run image_name COMMAND

Example: docker run -ti ubuntu bash

Note (1) : The tag "latest" is automatically used if no tag is specified.
Note (2) : If the image is not locally available, then it will be downloaded.
Note (3) : To exit from container, press CTRL+D or type 'exit'.

4. Useful arguments with the docker run command

1. -t : Run in a terminal
2. -I : Interactive mode
3. --rm : Run and then, delete this container when it exits.
4. -d : Detached . To run container and leave it running in the background. 

5. See all running containers

docker ps

6. See the last container that was run/exited.

docker ps -l

Note: Containers are not deleted automatically .

7. See all stopped containers list.

 

docker ps -a

8. Commit the changes you made to the container.

docker commit container_id (or NAMES . Get that using docker ps -l command)

Note: This does not change the container specifies her , BUT it creates a new image out of it.
The id of this new IMAGE , is returned to us after the commit.
Run "docker images" to see this new image in the list.

9. Add a tag to an image.

docker tag image_id some_name

10. Commit changes and tag container together.

docker commit container_id image_name

11. Working on a detached container.

docker attach container_id or name

Note: To exit (but not kill . Just leave it again running in the background) , press the key sequence:
Ctrl + p Ctrl + q

12. Run some extra command/process on the running container

 

docker exec

Example: docker exec -ti container_id/name COMMAND

Note: Do the above on a new terminal. You probably are already in the attached state to the same container on a different terminal.

13. Stop a container.

docker container stop container_id

14. Delete/remove a container.

docker container rm container_id

Note: You can also use the Name.

15. Delete an Image

docker image rm <Image Name>

 

16. Remove all unused containers

docker container prune

 

Python unittest and exceptions: Writing custom exceptions, and catching them.

Generally in unit test, we match the responses with an expected value . But there are cases , where calling a method or passing an incorrect value (or type of value) may raise an exception . This exception might me a custom exception . Or the message or error code might me specific to the situation. So how to write test for the same.

1. Calling a method with incorrect type , raises an exception.

class MyTest(unittest.TestCase):
    def test_exception(self):
         with self.assertRaises(xmlrpclib.Fault) as cm:
            self.Proxy.setParameter('name', 1234)
            self.fail('Expected exception of type xmlrpc.Fault')

Send email with excel file as attachment

We want to send a mail which will have some content in it's body and an attachment of excel file. The excel file is not physically saved on anywhere. We have just created it, filled it and and now , we want to send it in an email.

So what we really have as an attachment is a workbook object.

Let's get started.

    from io import BytesIO
    output = BytesIO()
    attachment.save(output)
    msg = EmailMultiAlternatives(subject, "", from_email, to_email, cc=cc, bcc=bcc)
    msg.attach('my_file.xlsx', output.getvalue(), 'application/vnd.ms-excel')
    msg.attach_alternative(content, "text/html")
    try:
        msg.send()
        success = True
    except socket.gaierror:
        success = False
    return success

Nose test command line tricks

If you are using nose plugin to run your tests , there are some handy tricks that we can use while running tests.

Run test and generate a XML report:

nosetests --with-xunit test_folder --xunit-file=nosetests_test_results.xml

Django: Creating PDF , writing content and returning as a downloaded file

Hi guys,

It took me hours to make it finally work but working with PDFs is a pain. I want to share what worked for me.

The canvas should be thought of as a sheet of white paper with points on the sheet identified using Cartesian (X,Y) coordinates which by default have the (0,0) origin point at the lower left corner of the page.

from reportlab.pdfgen import canvas
import io

pdf_buffer = io.BytesIO()
p = canvas.Canvas(pdf_buffer)
p.drawString(0, 100, "Hello People")
p.showPage()
p.save()

response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'attachment; filename="some_file.pdf"'
pdf_value = pdf_buffer.getvalue()
response.write(pdf_value)
return response

Above code , generates a pdf with the words "Hello People" written in it. 

PyCharms Handy short cuts

Hi friends. Today we are going to see some extremely useful (and must know) shortcuts for PyCharms .

Moving to last working point:

Ctrl+Shift+Backspace