PyCon 2017 summary

PyCon 2017 Summary

 

 

  • Science

    • The usage of Python in science was a major theme
      • Science is not just using Python & Open-Source more but also adopting their principles & ethos
      • Scientists understand the power of not just publishing results but also releasing the code & data that allow to reproduce it
      • The power of the Jupyter notebook & the plethora of Python science libraries were common in almost all talks
    • Jake Vanderplas keynote
      • offered a fascinating overview of this change & the reasons scientists like Python so much
      • Like others in the conference, Jake also repeated the old mantra of productivity > performance (didn’t Ruby started this decades ago?) & said that this is how he responds to people about performance:
        DANHn44UMAA2OXB
      • You may think that the nature of his “commute” is very short & exploratory, but he actually mentioned very large-scale projects he’s working on, such as the code for processing the images of the new LSST telescope (producing 30-giga-pixel images every 2 seconds) which is written in Python & available on GitHub
    • Katy Huff keynote
      • showed also how developers can have impact on the world’s most burning issues, by helping out the open-source projects that try to address them
    • Many other talks gave very cool examples, such as:

 

 

  • Best practices

    • Security

      • A nice talk by members of the security team at Docker provided a comprehensive explanation on how to do Threat Modeling
    • Documentation

    • API design

      • ApiStar
        • The creator of the Django REST Framework described his beautiful (i.e., simple & powerful) new framework – ApiStar – which allows you to create a REST/WebSockets/CLI API from plain python functions, by just having these functions properly describe their inputs & outputs.
        • This looks very promising & we should really try to use it, once it reaches maturity (which according to the author is when he’ll replace the README with proper docs)
      • 2 talks gave many tips & ideas on how to build good API’s
        • Key takeaways were
          • Besides usability, UX mainly means “How does the thing make you feel?”
          • The example of how the requests library simplifies the verbose code required to access some Web API using urllib
          • Good UX also
            • reduces mistakes
            • makes complex tasks easy & routine
            • drives adoption of your API
          • Good abstraction aligns with good UX
          • Write a Press-release 1st & README second
          • Build less structure up front
            • Because your domain knowledge isn’t enough when you start
    • Code review

 

  • Architecture & DevOps

    • Deployment

      • AWS
        • A practical talk explained in detail how to automate your AWS deployments & usage
      • Kubernetes
        • A truly hilarious keynote by Kelsey Hightower demonstrates the power of Kubernetes & a conversational interface to control it
    • Microservices

      • A nice overview of how a New York startup switched to Micro-services. One of their strong recommendations was to use the Mantl framework by Cisco
      • Another talk described the use of Amazon lambda functions to replace parts of your data-processing pipeline
    • Processing pipelines

      • Twilio described in details how they use Kafka as their central hub, & how much value they are gaining from this
        • Key takeaways were
          • It’s important to enforce schama (at the producer) – they use Arvo
          • Have a Metadata Registry, so all parties can check the schema for a topic
          • Have a REST API to serve the different producers
          • Use 2 services for storing Kafka topics to S3, which enables the replay of inputs
          • Have correctness checkers for validation in multiple stages
          • They use multiple Data-warehouse (using Redshift) & Ad-hoc analytics (using Jupyter notebook & Apache Spark) that load data from S3 & process it
      • Yelp presented how they built their stream processing pipeline

 

  • Python

    • General

    • Async

      • Testing
        • The author of Twisted described how to properly test async code, which is very prone to issues due to events arriving not in their expected order
      • Language

 

 

  • Also

    • You’d really want to see this hilarious Lightning “Talk” titled:

 

Advertisements