Category: core python

Following topics are covered

1. Introduction
2. Logging
3. Exception Handling
4. Searching & Regex
5. Packaging
6. File handling & problem solution

Introduction

Logging:

Basics of using the logging module to record the events in a file are very simple.
For that, simply import the module from the library.

1. Create and configure the logger. It can have several parameters. But importantly, pass the name of the file in which you want to record the events.
2. Here the format of the logger can also be set. By default, the file works in append mode but we can change that to write mode if required.
3. Also, the level of the logger can be set which acts as the threshold for tracking based on the numeric values assigned to each level.
   There are several attributes which can be passed as parameters.
4. The list of all those parameters is given in Python Library. The user can choose the required attribute according to the requirement.

After that, create an object and use the various methods as shown in the example.

#importing module
import logging
#Create and configure logger
FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(levelname)s %(message)s'
logging.basicConfig(filename="newfile.log",format= FORMAT,filemode='w')
#Creating an object
logger=logging.getLogger()
#Setting the threshold of logger to DEBUG
logger.setLevel(logging.DEBUG)
#Test messages
logger.debug("Harmless debug Message")
logger.info("Just an information")
logger.warning("Its a Warning")
logger.error("Did you try to divide by zero")
logger.critical("Internet is down")

The above code will generate a file with the provided name

Configuration:

Standard attributes names:

Logging Variable Data

1. this can actually be done directly by using a format string for the message and appending the variable data as arguments

import logging

name = ‘John’

logging.error(‘%s raised an error’, name)

Lazy Logging:

Issue:

log.debug(“I found {} and {}”, getone(), gettwo());

Seems, pretty good. A debug log with two parameters into the String.

Resources:

1. https://docs.python.org/3/howto/logging-cookbook.html
2. https://realpython.com/python-loggin/

Exception Handling:

syntax error-

missing : after ‘if’ or ‘for’

Exception :

Errors detected during execution are called exceptions

Handling Exceptions:

The try statement works as follows.

• First, the try clause (the statement(s) between the try and except keywords) is executed.
• If no exception occurs, the except clause is skipped and execution of the try statement is finished.
• If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.
• If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.

Syntax-

try:
x = int(input("Please enter a number: "))
break
except ValueError:
print("Oops!  That was no valid number.  Try again..

Note :Above program expects an integer ,If provided any other datatype it complains/Throws Error.

A try statement may have more than one except clause, to specify handlers for different exceptions. At most one handler will be executed. Handlers only handle exceptions that occur in the corresponding try clause, not in other handlers of the same try statement. An except clause may name multiple exceptions as a parenthesized tuple, for example

 except (RuntimeError, TypeError, NameError):

Order or evoking exceptions:

A class in an except clause is compatible with an exception if it is the same class or a base class thereof (but not the other way around — an except clause listing a derived class is not compatible with a base class). For example, the following code will print B, C, D in that order:

class B(Exception):
 pass
class C(B):
 pass
class D(C):
 pass
for cls in [B, C, D]:
 try:
   raise cls()
 except D:
  print("D")
 except C:
     print("C")
 except B:
 print("B")

Note that if the except clauses were reversed (with exceptB first), it would have printed B, B, B — the first matching except clause is triggered.

Summary: an except clause can capture only those exception class which are Base(or smaller footprint than that of raise exception class)

Supreme Exception Handler:

The last except clause may omit the exception name(s), to serve as a wildcard. Use this with extreme caution, since it is easy to mask a real programming error in this way! It can also be used to print an error message and then re-raise the exception (allowing a caller to handle the exception as well):

import sys
try:
   f = open('myfile.txt')
   s = f.readline()
   i = int(s.strip())
except OSError as err:
   print("OS error: {0}".format(err))
except ValueError:
 print("Could not convert data to an integer.")
except:
 print("Unexpected error:", sys.exc_info()[0])
 raise

The try … except statement has an optional else clause, which, when present, must follow all except clauses. It is useful for code that must be executed if the try clause does not raise an exception. For example:

for arg in sys.argv[1:]:
 try:
 f = open(arg, 'r')
 except OSError:
   print('cannot open', arg)
else:
      print(arg, 'has', len(f.readlines()), 'lines')
 f.close()

The use of the else clause is better than adding additional code to the try clause because it avoids accidentally catching an exception that wasn’t raised by the code being protected by the try … except statement.

Regular Expression module:

re — Regular expression operations

Both patterns and strings to be searched can be Unicode strings as well as 8-bit strings. However, Unicode strings and 8-bit strings cannot be mixed to match a literal backslash, one might have to write ‘\\\\’ as the pattern string, because the regular expression must be \\, and each backslash must be expressed as \\ inside a regular Python string literal.

The solution is to use Python’s raw string notation for regular expression patterns; backslashes are not handled in any special way in a string literal prefixed with ‘r’. So r”\n” is a two-character string containing ‘\’ and ‘n’, while “\n” is a one-character string containing a newline

search() vs. match()

Python offers two different primitive operations based on regular expressions: re.match() checks for a match only at the beginning of the string, while re.search() checks for a match anywhere in the string.

Example:

>>> re.match("c", "abcdef")  # No match
>>> re.search("c", "abcdef") # Match

however that in MULTILINE mode match() only matches at the beginning of the string, whereas using search() with a regular expression beginning with ‘^’ will match at the beginning of each line.

>>> re.match('X', 'A\nB\nX', re.MULTILINE)  # No match
>>> re.search('^X', 'A\nB\nX', re.MULTILINE)  # Match

FINDING all Adverbs

For example, if one was a writer and wanted to find all of the adverbs in some text, he or she might use findall() in the following manner:

>>> text = "He was carefully disguised but captured quickly by police."
>>> re.findall(r"\w+ly", text)
['carefully', 'quickly']

Use case: parsing log file

import sys
import re
logFileName = open(sys.argv[1], "r")
putputFileName = "parsedLogOutput.txt"
outputfile = open(putputFileName, "w")
lines = logFileName.readlines()
regex = "2019-08-12 03:00:*"
compiledReg = re.compile(regex)
for line in lines:
   if re.match(compiledReg, line, flags=0):
        outputfile.write(line + "\n")
  print(line)

Package and making a module deliverable

Each package in Python is a directory which MUST contain a special file called __init__.py. This file can be empty, and it indicates that the directory it contains is a Python package, so it can be imported the same way a module can be imported.

Example:

we create a directory called foo, which marks the package name, we can then create a module inside that package called bar. We also must not forget to add the __init__.py file inside the foo directory.

To use the module bar, we can import it in two ways:

import foo.bar

Tip:Listing functions of module:

# re is a module and following is script to list all functions:

Import re
find_members = []
for member in dir(re):
    if "find" in member:
        find_members.append(member)
print(sorted(find_members))

Problem Specific code snippets:

1.Run a linux command

 make a command
makecmd = "ls -lrt"
# Execute the command
cwd = subprocess.Popen(makecmd, stdout=subprocess.PIPE, shell=True)
(output, err) = cwd.communicate()
# print the output
print("output: " + str(output))

2. File Handling
   1. Text File
   2. Excel File

There are different modules for handling excel file like :

XlsxWriter(only for writing),openpyxl module(rw all variation of excels), xlwt module, panda module

Working with xlwt- in pyhton

Step 1-Install module:

pip install xlwt

Step 2-code snippet to write to an xml and apply style:

# importing xlwt module
import xlwt
workbook = xlwt.Workbook() 
sheet = workbook.add_sheet("Sheet Name")
# Applying multiple styles
style = xlwt.easyxf('font: bold 1, color red;')
# Writing on specified sheet
sheet.write(0, 0, 'SAMPLE', style)
workbook.save("sample.xls")

Working with panda for excel handling-

Python Pandas is a data analysis library. It can read, filter and re-arrange small and large datasets and output them in a range of formats including Excel.

Note: If only purpose is writing to excel use native module instead of pandas which internally uses it as it will be over-burden.

Pandas writes Excel files using the XlsxWriter modules

So first needs to install xlsxWriter:

Pip install xlsxWriter