Jupyter notebook and IPython first steps.

In this post, I am going to go through some initial steps for people who are looking to try out Jupyter Notebooks (JN). We will see how to:

• Use a venv in our Jupyter notebook to work on projects.
• Shortcuts for working with JN.
• Basic ways of using new python shortcuts provided in IPython.

Que.  How to install IPython inside a virtualenv and use it?

Ans: These are a few ways to launch the python installed in your venv

- venv/bin/ipython

- python -m IPython

Mongo DB first take

Testing new waters. My first NoSQL db was InfluxDB. It is a Time based No SQL type db. 

This time I wanted to taste the all famous MongoDB. I want to keep the basics handy. So I write as I learn.

Basic Commands ☝

Launch MongoDB:

~ mongo

See existing databases

show dbs

Grafana with Influxdb and python client

Recently I came across this tool called Grafana. Its a tool that you can use to show graphs and statistics of all kinds. The most popular combination for setup , is Grafana with influxdb as a database. 

I am sharing my setup steps, and supporting python scripts that can be used to generate information and write back to the database. And finally we get to see some nice graphs.

Note: I am using a Mac with Catalina OS 10.15. Python 3.6+.

Installation Plan:

1. Influxdb

2. Grafana

3. influxdb client module for Python.

• influxdb_client (for influxdb 1.8 or 2.x)
• influxdb (for 1.1.1 or less)
  

Docker : Most useful commands

Docker has a lot to talk about. But I am going to list a few commands here that come handy while working with it on a daily basis. We will list them in a sequence that looks like a tutorial.

1. List all docker containers

docker container ls

2. List all images

docker image ls

docker images

3. Spawn & run from an image, a new container.

docker run image_name COMMAND

Python unittest and exceptions: Writing custom exceptions, and catching them.

Generally in unit test, we match the responses with an expected value . But there are cases , where calling a method or passing an incorrect value (or type of value) may raise an exception . This exception might me a custom exception . Or the message or error code might me specific to the situation. So how to write test for the same.

1. Calling a method with incorrect type , raises an exception.

class MyTest(unittest.TestCase):
    def test_exception(self):
         with self.assertRaises(xmlrpclib.Fault) as cm:
            self.Proxy.setParameter('name', 1234)
            self.fail('Expected exception of type xmlrpc.Fault')

Send email with excel file as attachment

We want to send a mail which will have some content in it's body and an attachment of excel file. The excel file is not physically saved on anywhere. We have just created it, filled it and and now , we want to send it in an email.

So what we really have as an attachment is a workbook object.

Let's get started.

    from io import BytesIO
    output = BytesIO()
    attachment.save(output)
    msg = EmailMultiAlternatives(subject, "", from_email, to_email, cc=cc, bcc=bcc)
    msg.attach('my_file.xlsx', output.getvalue(), 'application/vnd.ms-excel')
    msg.attach_alternative(content, "text/html")
    try:
        msg.send()
        success = True
    except socket.gaierror:
        success = False
    return success

Nose test command line tricks

If you are using nose plugin to run your tests , there are some handy tricks that we can use while running tests.

Run test and generate a XML report:

nosetests --with-xunit test_folder --xunit-file=nosetests_test_results.xml

Django: Creating PDF , writing content and returning as a downloaded file

Hi guys,

It took me hours to make it finally work but working with PDFs is a pain. I want to share what worked for me.

The canvas should be thought of as a sheet of white paper with points on the sheet identified using Cartesian (X,Y) coordinates which by default have the (0,0) origin point at the lower left corner of the page.

from reportlab.pdfgen import canvas
import io

pdf_buffer = io.BytesIO()
p = canvas.Canvas(pdf_buffer)
p.drawString(0, 100, "Hello People")
p.showPage()
p.save()

response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'attachment; filename="some_file.pdf"'
pdf_value = pdf_buffer.getvalue()
response.write(pdf_value)
return response

Above code , generates a pdf with the words "Hello People" written in it. 

PyCharms Handy short cuts

Hi friends. Today we are going to see some extremely useful (and must know) shortcuts for PyCharms .

Moving to last working point:

Ctrl+Shift+Backspace

How and when to use *args and **kwargs in Python

args and kwargs are special arguments (the words itself are just convention) which can be passed to a function.

Python Special Methods : __call__

Python has some methods that are special in a way . If  these methods are created inside a class, they can bring in some unique behaviors . We will talk about the method '__call__' today.

Examples of Super method in Classes

Hi all. We are going to see some examples of usage of keyword 'super' here . 

Python Global Variables

The keyword "global" has a special purpose . If you look into the help docstring of this keyword, you must notice this line :

 It would be impossible to assign to a global variable without "global " 

The word to note is "Assign"

Now . In python , if you declare a variable outside the scope of all functions or classes, it is AVAILABLE GLOBALLY. Meaning I can use it anywhere anytime.

But what is NOT clear , is , you CANNOT change the value\contents of this variable.

Bringing statefulness into your program (e.x an integrating counter ), might need you to keep updating some variable's value . That means, functions must be able to change that value.

Hence, the keyword "global" .

Production server launch using Django and Apache and mod wsgi

Some of us develop Django applications which can be used inside Intranet for various purposes.
Some ideas are:

• HRMS apps
• Interview Management Tools
• Grievance Management Tool
• Financial Report Generators. 

These apps running on intranet, generally don't face too much traffic at a time, but they do need to compute or face heavy queries which is why, it's a good idea to release them on a proper environment . Typically they are light on UI. I choose to use bootstrap which is enough to achieve a decent look . 

Still you should serve them using something which is good at serving static content. So I chose Apache. Lets get started. 

Migrating your Django Project database from SQlite3 to MySQL

Django comes with an in-build default setup for SQLLite3 which is good for learning and minor static page based websites. Like blogs which is a standard example . But as soon you decide to create projects that might entail multiple users performing different operations at the same time, SQLLite is not recommended.

Hence the decision to migrate to a better DB system must be as early as possible. I had to under go a similar experience recently . It look a lot of time, trials to finally get it working . I am going to share you the complete steps assuming you don't even have a MySQL setup .

Quick guide to Logging to a file using Python

This is for setting up of a file logging system using Logging module

Step 1: Import the logging module.

import logging

Step 2: Create a logger for yourself. 

logger = logging.getLogger(__name__)

Working with context processors and WITH statement

We use. We forget. This is for myself so I remember the magic called Context Manager. You already must have used the "with statement" for file handling, threads, socket and other operations. I am going to talk about creating you OWN context managers.

The magic operator to be used is : @contextmanager. It can be imported from the module "contextlib"

To create a context manager, use the decorator contextlib.contextmanager

@contextlib.contextmanager
def getListCounts(x):
    try:
        from collections import Counter
        yield Counter(x)
    finally:
        print 'No of elements in the list : %d' % len(x)

Regular Expressions. Lets not be greedy.

Recently faced a situation when I wanted to extracted parts of a html code. But then I realized, I have no control over the amount of information regex is giving me back.

Python: Running executables using subprocess module

There are many ways to run a bat/exe file but the most widely used is the subprocess module.
And I would like to clearly show, how exactly to do that. When should you use what arguments.
I am going to specifically talk about the check_output module .

A better example of Class Inheritance in Python

Lots of people ask me, why should they use classes at all ? Well. I think its about control, and ease . It solves the headache of data sharing and accessibility. I am going to show you an example , which hopefully proves that classes and fundamentals like inheritance  are brilliant.