Part 9 Python Files and Resource Management

This post will cover reading and writing file. We will use built-in function “open()“. Arguments to be used in the open() function

  • file
  • mode
  • encoding – it is important that you specify encoding, otherwise you may get some errors.

import sys
sys.getdefaultencoding()

Example – Write
f = open(‘file.txt’, mode=’wt’, encoding=’utf-8′)
help(f)
f.write(‘this is text I wrote’)
f.write(‘this is the 2nd line \n’)
f.close()

Example – Read
f = open(‘file.txt’, mode=’rt’, encoding=’utf-8′)
f.read(32) # read 32 characters
f.read() # next line

#we should use
f.readline() # to return \n if there are still other lines
f.readlines() # it returns a list of each line

Part 8 Python Classes

You can get a long way with Python’s built in type, but if they are not fit for the job you want, this is when Class is needed.

Classes define the structure and behavior of objects and object class control its initialization. Class make complex problems tractable, can make simple solutions overly complex.

#airtravel.py
class Flight:
    def __init__(self, number):
        self._number = number

    def number(self):
        return self._number
    pass

from airtravel import Flight

f = Flight("TORO")
f.number()
  • Method – a function defined within a class
  • Instance methods – functions which can be called on objects
  • Self – the first argument to all instance methods
  • __init__() is an initializer, not a constructor.
  • self is similar to this in C++ or Java.
  • avoid name class with number(), we use _number

Part 7 Python Comprehensions Iterables Generators

Comprehensions is going through list, set, dict and doing some nice stuffs on those list.

Remember: [ expression(item) for item in iterable ]

we can do list comprehension instead of using for loop above.

Set comprehension eliminate duplicate

Remember: { expression(item) for item in iterable }

Dict comprehension

Remember: { key:value for item in iterable }

Reverse key-value pair by using dict comprehension.
This example define dict with x[0] as the key and x for value. ‘h’:’hi’ will be replaced by ‘h’:’hotel’.

List comprehension optional filter (this is good stuff).

[ expression(item) for item in iterable if function(item)]

Python Iterable and Iterator

Once you reach the end and if you call next() again, it will throw an exception.

Python Generator is an elegant function.

it will continue on the next yield when next ( ) is called.

Generator comprehension – similar to list but use ( ) instead.

( expression(item) for item in iterable )for item in iterable )

two generators are being called at the same time.

Part 6 Python Exception Handling

We can handle exceptions in Python to take appropriate actions in our programs when some thing hit an error. Below is how we handle exception.

def convert(s):
    """ Convert to an int
    try:
        x = int(s)
    except ValueError, TypeError:
        x = -1
    return x
this is more Pythonic way! – use raise keyword to throw error

If we know, what’s gonna throw an error, use ‘raise‘ keyword instead of try-except.

Part 5 Python Collections Advance

There are many collection types in Python and they are very useful to know and use them to a specific task. Here is the list:

  • Tuple
  • Str
  • Range
  • List
  • Dict
  • Set

Tuple

  • We use ( ) as to define tuple and separate items by comma.
  • We can access them by index like list.
  • We can use len( ) function to get the size.
  • We can iterate through them by using for loop.
  • We can concatenate them by using +.
  • We can repeat it my using * operator.
  • We can have nested tuple like this a = ( (1,2), (3,4) )
  • One item in a tuple we need to have command a = (1, )
  • We can omit ( ) when define a tuple a = 1, 2, 3, 4 this will be a tuple.
  • Tuple unpacking – we can use: var1, var2 = minmax(a)
  • Create a tuple from a list: tup1 = tuple( [1,2,3,4] ) this will be converted from list to tuple.
  • Create a tuple from string: tup2 = tuple(“Toro”)
  • Test member exist: var1 in tuple_a, var2 not in tuple_b

Str

  • len(s) to get size.
  • + to concatenate string: “Hello” + “Word”
  • join(s1, s2) is better than + to concatenate string. we can specify join string.
countries = ';'.join(['Thailand','Malaysia','Singapore'])
  • split(‘;’) use to split string: countries.split(‘;’)
  • common join string: ”.join([‘Toro’, ‘Test’, ‘Python’])
  • partition(‘word’) – use to divide string to 3 parts: “unforgetable”.partition(“forget”)
  • ‘_’ underscore – use to ignore the separator.
  • “String {0} and text {1}”.format(var1, var2) – use to format and inject value in string.
we use:3f to specify decimal point. use help(str) to learn more.

Range

Type of sequence of integer.

  • we supply a stop value: range(5)
  • we can supply start value: range(5,10)
  • it does not include stop value
  • we can provide step: range(1, 10, 2)
  • if we want a counter in loop, we use enumerate.
t = [1, 2, 3, 4, 5]
for i in enumerate(t):
    print(i)
# (0, 1)
# (1, 2)
# (2, 3)
# (3, 4)
# (4, 5)

for i, v in enumerate(t):
    print(v)

List

  • we use [ ] to define a list.
  • it can be index starting 0.
  • we can index starting from the end -1 by providing negative index.
  • we can slice index by providing start and stop location. It will not include stop.
s[start:stop]
  • Select all except the 1st and last one: s[1:-1]
  • stop index is option: s[start:]
  • select until stop: s[:stop]
  • all elements: full_slice = s[:] , this is a full copy of list.
  • x = s.copy(), this is the same above to full copy of list.
  • y = list(s), this is also the full copy but work with other type of collection.
the underlying list is the same, be careful. a & b is different.
  • Repeat list by s = [constant] * size
  • Be careful, when working with mutable list. It will refer to the same object.
  • use list.index(‘value’) to get an integer index of a particular value. if there is no value in that list, it will throw an error.
  • use list.insert(index_no, “value”) to insert a new value in a particular position.
  • list can be concatenate by using + or += operator.
  • use list.extend([a,b,c]) to concatenate as well. work the same.
  • list.reverse() – reverse list positions
  • list.sort() – sort A-Z
  • list.sort(reverse=True) – sort Z-A
  • we can sort by using a function.
we can use len function to sort string by len (A-Z).
  • sorted(list) – will return a new list without modifying the original list.

Dict

  • we define a dict by using { key: value}
  • we can also build a dict from ‘dict( )’ by passing pairs of key value.
  • we can extend a dict from another dict by using update( )
we can access dict by using key-value pairs
we can get only values of the dict without getting keys
we can access only keys

for key in colors.keys():
print(key)
  • in, not in keywords work only with key of the dict.
  • del(key) remove element by key.

Set

  • un-order collection
  • { } use to define set like dictionary but no key-value pair
  • d = { }, we will get dict
  • d = set() we will get set
  • no dup item
  • unique = set(list), we will get a unique item from dup items in list
  • s1.add(value), to add a new item to set
  • s1.update(list), to add multiple values on one go
  • s1.remove(item), to remove an item, but if it does not there, throw error.
  • s1.discard(item), to remove an item, but if it does not there no error.
  • set1.union(set2)
  • set1.intersection(set2)
  • set1.difference(set2), equal to in s1 but not s2
  • set1.symmetric_difference(set2), equal to in s1 or s2 but not intersect
  • set1.issubset(set2)
  • set1.issuperset(set2), to check if set1 bigger than set2
  • set1.isdisjoint(set2), to check nothing in common.

Part 4 – Python Objects

When we create a variable in Python, it will create an object and point the variable to reference that object. we can use id(var) syntax to get unique ID of the object. var1 is var2 syntax use to check if those objects are the same.

We need to keep in mind when we assign one variable to another because it is referencing the same object. When we change a value in that object, the other variable will get the change as well because it is the same object.

# comparison
list1 = [1,2,3]
list2 = [1,2,3]

list1 == list2  # True
list1 is list2  # False
Passing object reference to a function. It refers to the same object. If you want the function to not verify the argument, function has to copy the value instead.

Python has a scope in referencing variable. If we want a local binding to refer to a global variable define at the top of the module, we need to specify ‘global‘ keyword. It will refer to count=0 at the top.

In Python, everything is an object. Default argument expressions evaluate once, when def is executed.

Part 3 Python Module Class and Function

Module is the group of code which can re-use by others. As we have seen in previous parts, we use ‘import module_name‘ to use other modules or libs. We use ‘def’ keyword to define a module or function.

def function_name():
    print("DO THING")

def main():
    # START YOUR CODE

if __name__ = "__main__":
    # run main
    main()

we will not be able to run main( ) on REPL because above code did not allow us to pass URL in so we need to adjust the code a bit below. This is the best practice for organize functions and manage main( ) to run on both script and REPL.

Advance command line argument parsing:

Python Standard Lib: argparse

Third-party Lib: docopt

Docstrings

we use it to document our Python code. We use Google Python Style Guide for docstrings.

help(fetch_words)
# we will get a nice document on the terminal

We need to put comment at the top of your module to make it available on help( ) command
"""
    Explain how your module work
"""

Put below at the top of your script to tell Python know where it should use Python environment.It works on Windows too.
#!/usr/bin/bin/env python3

Part 2 Python Collections

There are many types of collections in Python that we can use to store values as a collection such as str, bytes, list, dict. Then, we will cover for loops.

str

str is immutable sequences of Unicode. Meaning string cannot be changed after it is declared. str support both single quote and double quote.

'your words'
"your words"

# Multi line string, we use triple quote.
"""
    This is line 1
    This is line 2
"""

# \n is a new line string
# \ is an escape string

On Windows, we will end-up headache with \ because Windows path use that one to refer to a folder. Python has a solution using raw string.

String can be access by using [ ] to get a character from a string.

s = 'Thailand'
s[4] 
# we will get l, Python index start from 0

We can use many methods of string. You can check by typing “help(str)“.

c = 'Thailand'
c.capitalize() # THAILAND

bytes

similar to str.

b'data'
b"data"

it supports similar methods as string. This is very useful when you are coding and there are some values in local languages not English. We need to encode Thai to bytes to store Thai characters and decode it when retrieve the value.

name = "My name is โตโร่"
data = name.encode('utf-8')

# when we use it
to_be_use = data.decode('utf-8')

# Files, Network Resources, HTTP responses are transmitted as bytes stream, so we may need to decode it before use.

List

mutable sequences of objects. We use [ ] to define a list.

a = []
a = list("characters")

# List has many useful methods
a.append('x')

Dict

mutable sequences of objects. We use { } to define a dict.

we can retrieve a value by referring to a key.

d = {'toro' : 123, 'jack' : 456}
d['toro'] # 123

# assign a new value
d['toro'] = 999

# add a new item by using a new key
d['new'] = 678

# dict does not guarantee the order

For loop

for item in collections:
    print(item)

# Example
d = {'toro' : 123, 'jack' : 456}
for key in d:
    print(key, d[key]) # print key-value pairs

Next part, we will learn about function or module of Python. Before we move on, here is a code example. It is a best practice that we use ‘with’ keyword to work with external resources like http request. HTTP returns bytes string.

>>> from urllib.request import urlopen
>>> with urlopen('https://jirath.in') as blog:
...     blog_words = []
...     for line in blog:
...         line_words = line.split()
...         for word in line_words:
...             blog_words.append(word)
...
>>> blog_words
[b'<!DOCTYPE', b'html>', b'<html', b'lang="en-US"', b'class="no-js', b'no-svg">'                                                                                        , b'<head>', b'<meta', b'charset="UTF-8">', b'<meta', b'name="viewport"', b'cont                                                                                        ent="width=device-width,', b'initial-scale=1">', b'<link', b'rel="profile"', b'h                                                                                        ref="http://gmpg.org/xfn/11">', b'<script>(function(html){html.className', b'=',                                                                                         b"html.className.replace(/\\bno-js\\b/,'js')})(document.documentElement);</scri                                                                                        pt>", b'<title>Tech', b'Savy', b'–', b'for', b'IT', b'profressional</title                                                                                        >', b'<link', b"rel='dns-prefetch'", b"href='//s0.wp.com'", b'/>', b'<link', b"r                                                                                        el='dns-prefetch'", b"href='//fonts.googleapis.com'", b'/>', b'<link', b"rel='dn                                                                                        s-prefetch'", b"href='//s.w.org'", b'/>', b'<link', b"href='https://fonts.gstati                                                                                        c.com'", b'crossorigin', b"rel='preconnect'", b'/>', b'<link', b'rel="alternate"                                                                                        ', b'type="application/rss+xml"', b'title="Tech', b'Savy', b'»', b'Feed"',

We can decode byte responses from HTTP by change the line of code below.

...         line_words = line.decode('utf-8').split()

Part 1 Getting Start with Python

We will use Python3 to start with. Before we can start, we need to install Python on our computer. Python can work on any OS: Windows, MacOSX, Linux.

You need to download at: python.org, and click Downloads.

When install, make sure you check “Add Python to PATH” if you use Windows. After installation, you can open Power Shell and type “python” to start to code!.

If you use MacOSX, it has a pre-build Python2.7, but I recommend you install Python3. After installation, you need to open a terminal and type “python3” to run Python3, and you are ready to code!.

If you use Linux Utuntu, it already has Python3 so you can open the terminal and type “python3“.

REPL = Read, Eval, Print, Loop. It refers to python interpreter. It will wait and read out input, interpret (evaluate), print the result, and loop to do again. Below is an example of code you can enter on REPL after you run “python3” command.

2 + 2
6 * 7
x = 5
3 * x
print("Hello World")

It will print the result on the screen. Some commands will not print the result, but store it on variables. If you want to exit the REPL, on Windows, Press CTRL+Z+Enter. If Mac or Linux, Press CTRL+D.

Python use 4 space as an indentation. It is not the same as Tab. So, when writing code. You need to indent it next line in a code block of 4 space.

for i in range(10):
    x = i * 2
    print(x)

Standard Library

We can use a standard library by using keyword “import name_of_module“. We can see documentation on how we can use that lib by using command “help(lib_name.function_name)” such as “help(math.sqrt)“.

There are 3 forms of import:

import math
# Usage
math.factorial(3)

from math import factorial
# Usage
factorial(3)

from math import factorial as fac
# Usage
fac(3)

I recommend use option 3 because it is easy to read and easy to use the lib.

Build-in data type in Python

int
# store integer such as 10

float
# store float such as 3.5

# we can convert float to int
int(3.5)  >>> 3
int(-3.5) >>> -3

3e8 >>> 300000000.0

#Convert to float
float(7)

None is a data type in Python

Bool has True and False
bool(0) >>> True
bool(1) >>> False
bool(-3) >>> False

bool(0.0) >>> True
bool(0.2) >>> False

bool([]) >>> False
bool([1,2,3]) >>> True

bool("") >>> False
bool("H") >>> True

Python comparing values (Relational Operators)

Condition Statement

we need to use condition to examine condition. The 1st one to use is IF statement.

if expression:
    DO SOME THING
elif expression2:
    DO SOME THING
else:
    DO SOME THING

#Example
if True:
    print("Yes")

Loop using while keyword

while expression:
    print("This is doing")

# Example
c = 5
while c != 0 :
    print(c)
    c -= 1     # this equal to c = c-1

# we can use 'break' keyword to stop the loop
while True:
    if expression:
        break
    print("Hey")

I hope you enjoy reading until this end. Next post will dig down to detail.

Python Fundamental – Introduction

Python is the back-end of high traffic side such as YouTube, Instagram, Reddit, and Dropbox. They use Python Web Framework to develop such as Django, Flask, Pyramid. Python is very popular in data science. There are many useful packages such as Astropy, Biopython, Numpy, SciPy. Python is the back-end for big cloud solutions and players such as Ansible, Boto3 for AWS, MS Azure SDK for Python. It is the center for analytics such as Pandas, Matplotlib, Bokeh, TensorFlow, ScikitLearn. With Python, we can do a lot of everything and it is very popular now with large communities. From beginner to professional, Python can give your ways to success. It is enjoy and fun to learn this language.

Who use Python?

There are several implementation of Python, but normally we use CPython.

There are two versions of Python; Python2 and Python3. We normally use Python3 to develop new applications because they will stop support Python2 soon so it is recommend that you start working on Python version 3.

Python provides standard libraries which we can use to build complex software without inventing a wheel so we can quickly use instant and they documentations are good.