help, inspiration, python, Uncategorized

Your first talk

Hi there folks. It’s been a long time since I wrote on this blog. I have been very busy with university applications. A lot has happened recently which I will love to share with you. Firstly, I got a news from a friend that my book is being used in McGill University to teach Python programming. That is something I have always wanted, Continue reading

Standard
Featured Image -- 658
Uncategorized

Generating Text Using a Markov Model

alexhwoods

Screen Shot 2015-08-05 at 10.57.18 AM

A Markov Chain is a random process, where we assume the previous state(s) hold sufficient predictive power in predicting the next state. Unlike flipping a coin, these events are dependent. It’s easier to understand through an example.

Imagine the weather can only be rainy or sunny. That is, the state space is rainy or sunny. We can represent our Markov model as a transition matrix, with each row being a state, and each column being the probability it moves to another.

f5f65f94c2f9cd72f9f2ce10499e6b7b However, it’s easier to understand with this state transition diagram.

220px-Markov_Chain_weather_model_matrix_as_a_graph

In other words, given today is sunny, there is a .9 probability that tomorrow will be sunny, and a .1 probability that tomorrow will be rainy.

Text Generator

One cool application of this is a language model, in which we predict the next word based on the current word(s). If we just predict based on the last word…

View original post 507 more words

Standard
Uncategorized

Optimise Python with closures

Wrong Side of Memphis

This blog post by Dan Crosta is interesting. It talks about how is possible to optimise Python code for operations that get called multiple times avoiding the usage of Object Orientation and using Closures instead.

While the “closures” gets the highlight, the main idea is a little more general. Avoid repeating code that is not necessary for the operation.

The difference between the first proposed code, in OOP way

and the last one

The main differences are that both the config dictionary and the methods (which are also implemented as a dictionary) are not accessed. We create a direct reference to the value (categories and mode) instead of making the Python interpreter search on the self methods over and over.

This generates a significant increase in performance, as described on the post (around 20%).

But why stop there? There is another clear win in terms of access, assuming that the…

View original post 310 more words

Standard
Uncategorized

Ultimate guide for scraping JavaScript rendered web pages

I really wanted to write a guide for this myself but didn’t get the time. Here Naren arya wrote a great post and I think that you should definitely give it a look.

IMPYTHONIST

We all scraped web pages.HTML content returned as response has our data and we scrape it for fetching certain results.If web page has JavaScript implementation, original data is obtained after rendering process. When we use normal requests package in that situation then responses those are returned  contains no data in them.Browsers know how to render and display the final result,but how a program can know?. So I came with a power pack solution to scrape any JavaScript rendered website very easily.

Many of us use below libraries to perform scraping.

1)Lxml

2)BeautifulSoup

I don’t mention scrapy or dragline frameworks here since underlying basic scraper is lxml .My favorite one is lxml.why? ,It has the element traversal methods rather than relying on regular expressions methodology like BeautifulSoup.Here I am going to take a very interesting example.I am so amazed after finding that ,my article is appeared in recent PyCoders weekly issue…

View original post 681 more words

Standard