Useful list of Python Resources

view raw
PYTHON_RESOURCES.md
hosted with ❤ by GitHub

Sponsored Post Learn from the experts: Create a successful blog with our brand new courseThe WordPress.com Blog

Are you new to blogging, and do you want step-by-step guidance on how to publish and grow your blog? Learn more about our new Blogging for Beginners course and get 50% off through December 10th.

WordPress.com is excited to announce our newest offering: a course just for beginning bloggers where you’ll learn everything you need to know about blogging from the most trusted experts in the industry. We have helped millions of blogs get up and running, we know what works, and we want you to to know everything we know. This course provides all the fundamental skills and inspiration you need to get your blog started, an interactive community forum, and content updated annually.

Presentation slides with Reveal.js

Using the reveal javascript library its possible to build intricate presentation slides for the browser. While its possible to create a sample using a web client over at https://slides.com/ I personally prefer to download the latest version directly from the github account Latest release in order to generate your slide presentation by hand.

Steps to get it all up and running:

 

  1. Create a new folder and extract the latest reveal.js code into a sub-folder called “reveal.js”
  2. In the root of your project copy the following html code into a file named index.html
  3. Open the index.html file with your browser of choice (note: chrome seems to be the preferred option)
<!– -*- indent-tabs-mode:t; -*- –>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title>Reveal Example Slides</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no"/>
<link rel="stylesheet" href="./reveal.js/css/reveal.css"/>
<link rel="stylesheet" href="./reveal.js/css/theme/night.css" id="theme"/>
</head>
<body>
<div class="reveal">
<div class="slides">
<section>
<h1>Reveal Example Slides</h1>
<h2>Welcome</h2>
</section>
<section data-markdown
data-separator="^\n\n\n"
data-separator-vertical="^\n\n"
data-separator-notes="^Note:"
data-element-attributes="{_\s*?([^}]+?)}">
<script type="text/template">
## Slide 1
<! .slide: id="slide1" >
<p class="fragment grow">grow</p>
<p class="fragment shrink">shrink</p>
<p class="fragment fade-out">fade-out</p>
<p class="fragment fade-up">fadeup (also down, left and right!)</p>
<p class="fragment current-visible">visible only once</p>
<p class="fragment highlight-current-blue">blue only once</p>
<p class="fragment highlight-red">highlightred</p>
<p class="fragment highlight-green">highlight-green</p>
<p class="fragment highlight-blue">highlightblue</p>
## Slide 2
<! .slide: id="slide2" datatransition="concave" >
Note: add diagram
</script>
</section>
</div>
</div>
<script src="reveal.js/lib/js/head.min.js"></script>
<script src="reveal.js/js/reveal.js"></script>
<script>
Reveal.initialize({
controls: true,
progress: true,
history: true,
center: true,
transition: 'none', // none/fade/slide/convex/concave/zoom
dependencies: [
{ src: 'reveal.js/lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'reveal.js/plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'reveal.js/plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'reveal.js/plugin/highlight/highlight.js', async: true, condition: function() { return !!document.querySelector( 'pre code' ); }, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'reveal.js/plugin/zoom-js/zoom.js', async: true },
{ src: 'reveal.js/plugin/notes/notes.js', async: true }
]
});
</script>
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? 'reveal.js/css/print/pdf.css' : 'reveal.js/css/print/paper.css';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
</body>
</html>

view raw
revealExample.html
hosted with ❤ by GitHub

The above sample is just a quick setup to get you going but there are many features in this library that you can take advantage of:

  • Markdown support using “<section data-markdown> …”
  • PDF support by going to “?print-pdf” in chrome. The sample above provides support for this using a small javascript tag which makes use of some inbuilt javascript libraries.
  • Hidden notes support using “<section data-separator-notes=…>”
  • Slide data transitions such as “fading” in and out, zooming as well as controlling the speed

There are many more features that you can play about with including embedded videos, images and other multimedia. One feature that particularly interests me is embedding external markdown files. In terms of project development teams would be able to provide customer demos along side your versioned code base. It’s also possible to override the existing css but you may find this to be difficult when using the markdown content due to the way that this is converted by reveal.js on the fly.

For a quick introduction on other features in the current version 3 release have a look at the following slideshare from the author of Reveal.js Hakim El Hattab:

Creating a Gists Webpage With ReactJS

I decided to build a simple ReactJS webpage to render my personal Gists from GitHub as an basic kata.

However while the ReactJS part was fairly straightforward I was once again reminded about Cross Site Scripting (XSS) issues when trying to call API’s from another domain using javascript. Thankfully JQuery jsonp still works:

componentDidMount(){
    $.ajax({
      url: "https://api.github.com/users/m-x-k/gists",
      jsonp: "$jsonp"
    }).done(function(data) {
      console.log(data);
      this.setState({gists: data});
    }.bind(this));    
}

Obviously in a production situation this isn’t what we would rely on. Reverse proxies like Nginx, HAProxy or Apache are much better suited to revolve these types of issues. Ideally we would setup a reverse proxy on “api.github.com” so that we can share a common domain for the client side requests. For example if your web page url was http://my.gists.com/index.html you might setup a reverse proxy like http://my.gists.com/api/m-x-k/gists.
Note: while its possible to mitigate the problem with headers it’s not an ideal solution as it depends on browser support.

Rapid prototyping with code templates

Over the last few years I have been using Python quite a lot to build rapid prototypes. By far the best command line tool I’ve found for this is the python ‘cookiecutter’ project below:

https://github.com/audreyr/cookiecutter

There are a large number of sample cookiecutter projects out there but it makes sense to build and maintain your own if you want to get the latest setup.

For ease of use I have a number of linux command line aliases to create new projects from my own code template projects:

alias new-springboot-app='cookiecutter https://github.com/m-x-k/cookiecutter-spring-boot'
alias new-python-app='cookiecutter https://github.com/m-x-k/cookiecutter-python-flask'
alias new-react-app='cookiecutter https://github.com/m-x-k/cookiecutter-react'
alias new-java-app='cookiecutter https://github.com/m-x-k/cookiecutter-java'
alias new-elm-app='cookiecutter https://github.com/m-x-k/cookiecutter-elm'
alias new-scala-app='cookiecutter https://github.com/m-x-k/cookiecutter-scala'
alias new-golang-app='cookiecutter https://github.com/m-x-k/cookiecutter-golang'
alias new-play-app='cookiecutter https://github.com/m-x-k/cookiecutter-play'

In some cases these code template projects have been customised to my own preferences with IDE support etc … but for the most part they tend to be fairly vanilla in case other people might find them useful and also to allow me to customise each new project. So for example with Java applications I tend to prefer IntelliJ with Gradle but beyond that for me it doesn’t make sense to preload all of your favourite java libraries.

In some work places I have noticed a tendency to try and standardise code structure with similar code template tools (e.g. maven archetypes). In many cases this can save project setup time but I tend not to favour this approach. Ideally in agile development teams it is best to reach a group consensus based on the project requirements first and then trying to agree on code standards such as naming, structure etc … A good place to start is often Uncle Bob’s Clean coders book which is widely distributed and possibly combining this with Extreme Programming techniques such as TDD, Pair Programming or Mobbing. This combined approach often gets extremely good results allowing for a wider group consensus and shared ownership.

However in many situations the first place to start on a project is a quick prototype. It can therefore be convenient to use code templates in these situations. This actually helps to reinforce the concept that code implemented as part of the prototype is ‘temporary’ and therefore should be discarded and reimplemented by the team from scratch.

Building a responsive photo gallery

Recently I needed to build a photo gallery for a website built with ReactJS. I came across a convenient react library “react-photo-gallery” which was ideal for this purpose. As a proof of concept I built a small Spring Boot app with ReactJS support:

https://github.com/m-x-k/photo-gallery

The application contains a few sample images which can easily be orientated to fit various devices and screen sizes for responsive behaviour.

I also needed to provide a demo to show off some of the functionality before implementing further. Luckily Heroku provides a great environment for sample demo sites. As usual Heroku can be a little slow on the first load as it builds the app on demand. You can reach the demo site below:

https://responsive-photo-gallery.herokuapp.com/

The code implementation was straight forward to setup with the project split into two separate folders. One for the spring boot ‘src’ code and the other ‘ui’ folder for the ReactJS code. Support for Docker has been included as well as details and scripts on to upload the app and any photos to a Heroku account. The only tricky part was getting the folder permissions just right for the photos to load successfully.

At the minute I tend to favour Gradle for building Java projects it provides a robust DSL with various plugins and support for Maven etc … When it comes to Javascript building I tend to favour NPM/Yarn with Webpack especially for the ease of setup when it comes to ES6 transpiling. The combination allowed me to quickly setup an application which could allow the full stack to be built on demand.

Web scraping with Python

Sometimes all you need is to grab some simple content from a web page just to get the job done. In my case I wanted to get a list of h2 content tags from a remote website. The following is an example script using the Python BeautifulSoup library:

import fire
import requests
from bs4 import BeautifulSoup
# Web scrape example using python and BeautifulSoup
# Setup: pip install requests beautifulsoup4 fire
# Run: python webscrape.py scrape
class WebScrape(object):
url = "https://www.themuse.com/advice/43-simple-habits-thatll-improve-your-life-even-if-you-just-pick-one&quot;
def scrape(self):
response = requests.get(self.url, verify=False)
if response.status_code == 200:
data = response.text
lines = BeautifulSoup(data, "html.parser").find_all("h2")
for line in lines:
print(line.text)
else:
return "ERROR: %s" % response.text
if __name__ == '__main__':
fire.Fire(WebScrape)

view raw
webscrape.py
hosted with ❤ by GitHub

Reactive programming in Python

RxPy

Examples based on Reactive Python for Data Science video tutorials by Thomas Nield.

 

Simple subscriber example:

from rx import Observable, Observer
letters = Observable.from_(['Alpha', 'Beta', 'Gamma', 'Delta', 'Epsilon'])
letters.subscribe(on_next=lambda s: print(s),
on_completed=lambda: print('Done'),
on_error=lambda e: print(e))

view raw
rx_simple.py
hosted with ❤ by GitHub

Using operators:

from rx import Observable, Observer
letters = Observable.from_(['Alpha', 'Beta', 'Gamma', 'Delta', 'Epsilon']) \
.map(lambda s: len(s)) \
.filter(lambda i: i >= 5) \
.subscribe(lambda i: print(i))

view raw
rx_operators.py
hosted with ❤ by GitHub

Intervals:

from rx import Observable, Observer
Observable.interval(1000) \
.map(lambda i: "{0} Mississippi".format(i))\
.subscribe(lambda s: print(s))
input("Press any key to quit")

 

Seven Deadly Sins of Microservices

The Microservice approach is gaining a lot of traction at the minute however not all of it is smooth sailing for development teams. A number of anti-patterns have been suggested in the following article and video recording:

https://opencredo.com/the-seven-deadly-sins-of-microservices-redux/

https://www.infoq.com/presentations/7-sins-microservices

Personally I have to admit to observing several of these issues first hand. In the case of the Sloth sin there is a natural tendency for some teams to enforce DRY principles to the extreme. One such example of this is the desire to create a single common domain to be shared across Microservices. I personally feel that this should be avoided at all costs as it results in high degrees of efferent coupling across applications. Any benefits this approach might provide will ultimately work against the Microservice desire to encapsulate and segregate common functionality. In addition the net effect will often require bigger releases to accommodate basic changes to the domain.

Thinking in Coroutines (Python Asyncio)

Interesting O’Reilly podcast today by Lukasz Langa (Facebook) on Python Asyncio.

Python threads still seems to suffer when it comes to parallel processing although this all seems to be due to the structure of the Global Interpreter Lock.

Thankfully asyncio looks reasonably developer friendly by focusing on concurrency rather than parallel processing. Combined with the use of Processor Pool Executors it looks like some potential performance gains. It also finally suggests some real gains to finally switch up to Python 3.5!

Spring Rest Exception Handling

 

compile('org.springframework.boot:spring-boot-starter-web')

view raw
build.gradle
hosted with ❤ by GitHub

@RestController
public class GreetingController {
@RequestMapping("/my/rest/exception")
public @ResponseBody ResponseEntity<String> myException() {
if (true)
throw new MyRestException("should see me");
return new ResponseEntity<String>("don't see me", HttpStatus.OK);
}
}

public class MyRestException extends RuntimeException {
public MyRestException(String s) {
super(s);
}
}

view raw
MyRestException.java
hosted with ❤ by GitHub

@ControllerAdvice
public class MyRestExceptionHandler extends ResponseEntityExceptionHandler {
@ExceptionHandler(value = {MyRestException.class})
protected ResponseEntity<Object> handleConflict(RuntimeException ex, WebRequest webRequest) {
String bodyOfResponse = "Well that's another fine mess you got me into";
return handleExceptionInternal(ex, bodyOfResponse, new HttpHeaders(), HttpStatus.CONFLICT, webRequest);
}
}

Hosting Static Website on AWS S3 Bucket

Great little introduction on how to host a static website on AWS S3 bucket.

http://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-custom-domain-walkthrough.html

Its really easy to setup and configure. I used a GoDaddy certificate and redirected with a masked url. All in all it took a matter of minutes to set it up but you have to make sure to make the site public each time you upload / re-upload content to the s3 bucket 😦

Spring Boot REST External APIs

Recently I’ve been developing a lot of REST styled microservices using Spring Boot. We’ve been looking a lot at technologies to make these new endpoints available externally.

Some nice technologies we’ve come across so far include:

  • Spring Cloud Security
    • https://cloud.spring.io/spring-cloud-security/
    • Greatly simplifies spring security 4 so that all you need are meta annotations and basic spring configuration (i.e. application.yml).
    • This allowed us to setup microservices as Resource Services!
  • JSON Web Tokens (JWT)
    • Supports json data in authentication tokens (i.e. bearer token)
    • The http://jwt.io provides a fantastic introduction and a very useful debugging tool
  • Swagger
    • Rest api documentation generated at source
  • HTTPie