Create simple and powerful spiders or crawlers with pyscalpel!
Nowadays, there are many usages where we need to scrape information from web pages like news feed, beautiful link rendering with image and text like social media do, research for data scientists, etc..
So it is important if there is a framework that help us to do this with little effort. The most known in the python ecosystem is probably scrapy. It is a veteran using the Twisted asynchronous backend (note that it has recently add partial support for asyncio). But I feel like its usage was heavy. We have to…
Write simple and complex CLI applications in python using click.
As developers, we often used command line interface (CLI) applications for different purposes like writing files with vim or emacs, running containers with docker, researching a pattern in any file using grep, etc… When I began programming, I have always wondered how do we write one? Then after some researchs, I found click. It was created by the same author of Flask, Armin Ronacher, who was frustrated to not find a suitable library to write Flask CLI application.
Before diving into the usage of click, I want to present you…
A structured concurrency library that makes your live easy!
These last years there has been a new trend for concurrency in python called asyncio. It’s goal is to help us create more efficient concurrent programs than the ones we traditionally created with threads. It leverages the new keywords async/await introduced in python3.5 and achieve better concurrency through the usage of coroutines sometimes defined as lightweight threads.
Don’t be afraid of pythons, you can handle them!
Until recently I never had the issue of managing multiple python environments since I was always working with the default python2 or python3 installed on my Ubuntu laptop. But with my new job, things have changed, we have different projects using different versions of python, we use python2 and different versions of python3 and I also want to install the pypy interpreter that I’ve heard the most good about.
And that’s when the trouble started. Trying to compile pypy or a newer version of python3 was not successful. For the former…
If you are new in python programming, sometimes you feel overwhelmed when it comes to manage package dependencies and publish your project on pypi or other form of package registry. There are too many tools involved in the ecosystem: virtualenv, pip, pipenv, flirt, twine, etc.. This is a bit frustrating because you don’t know what to use to begin and you very often have to juggle between several tools to get to your end.
Fortunately a new tool has arrived and solved all these issues, it is called poetry! This tutorial will be based on poetry 1.0.0 …
Its main advantage over tox for me is that you can take advantage of all the power of the python language in your automation process, but it has others cool features. Also it is used by some great projects like google-cloud-python or urllib3.
For the example we are going to imagine a simple project with a module example.py that will have one function adding two numbers.
def addition(a, b): return…
Déserteur camerounais résidant désormais en France. Passionné de programmation, sport, de cinéma et mangas. J’écris en français et en anglais dû à mes origines.