GET /api/v2/video/1071
HTTP 200 OK Vary: Accept Content-Type: text/html; charset=utf-8 Allow: GET, PUT, PATCH, HEAD, OPTIONS
{ "category": "EuroPython 2011", "language": "English", "slug": "experiences-making-cpu-bound-tasks-run-much-faste", "speakers": [ "Ian Ozsvald" ], "tags": [ "cython", "git", "multiprocessing", "network", "numpy", "nvidia", "profiling", "pycuda", "runsnakerun", "tutorial", "windows", "wxpython" ], "id": 1071, "state": 1, "title": "Experiences making CPU-bound tasks run much faster", "summary": "[EuroPython 2011] Ian Ozsvald - 22 June 2011 in \"Training Pizza Margherita \"\n\n", "description": "UPDATE - post-event I've created a [49 page PDF write-\nup](http://ianozsvald.com/2011/06/29/high-performance-python-tutorial-v0-1\n-from-my-4-hour-tutorial-at-europython-2011/) which summarises the 4 hour\ntutorial\n\nAs a long-time R&D consultant I'm often working to make slow, experimental\ncode run faster for tasks like physics simulation, flood modeling and natural\nlanguage processing. Python allows a smooth progression from rough-and-ready\n(but slow) algorithms through to finely tuned tasks that efficiently use as\nmuch CPU power as you can bring to bear. Speed-ups of 10-500* can be expected\nfor the Mandelbrot code we'll use.\n\nIn this talk I'll cover a set of libraries that make CPU-bound tasks run much\nfaster. We'll begin with a look at profiling using RunSnakeRun and\nline_profiler to identify our bottleneck. We'll take a look at slow algorithms\nin Python and how they can run faster using numpy and numexpr.\n\nNext we'll cover the use of multiprocessing to utilise multiple CPU cores\nalong with Cython or ShedSkin to easily use C code in a friendly Python\nwrapper. Multiprocessing on a quad-core system can often provide a 4* speed-up\nfor the right tasks. Next parallelpython will let us run our code on a network\nof machines.\n\nFinally we'll look at pyCUDA to utilise an NVIDIA GPU. CUDA can give the best\nimprovements for mathematical problems (over 100* on the right tasks) but\nworks on a narrower set of problems.\n\nHow it'll work: The tutorial will be hands on, you'll be converting example\nfiles from normal Python to faster variants using the tools below. All of it\nis optional, you'll get the most benefit by having everything installed. We'll\nwork in groups and open discussion is encouraged.\n\nNOTE - you are expected to have all these tools installed _before_ the\ntutorial (if you don't, you might find it hard to follow what's going on!).\n\nI'll be using Python 2.7.1 on a Macbook (Snow Leopard). All of these tools run\non Windows and Linux, as long as your versions are fairly recent everything\nshould run just fine.\n\nMy versions (roughly ordered by importance):\n\n * Python 2.7.1\n * RunSnakeRun 2.0.1b6 (with wxPython 2.8.12.0 Unicode)\n * line_profiler (1.0b2)\n * Cython 0.14.1\n * ShedSkin 0.7.1\n * numpy 1.5.1\n * numexpr 1.4.2\n * ParallelPython 1.6.1\n * pyCUDA HEAD from git as of 14th June 2011 (with CUDA 4.0 drivers)\n * PyPy 1.5\n\nSome background reading:\n\n * [http://ianozsvald.com/2010/07/14/22937-faster-python-math-using-pycuda/](http://ianozsvald.com/2010/07/14/22937-faster-python-math-using-pycuda/)\n * [http://ianozsvald.com/2008/11/17/making-python-math-196-faster-with-shedskin/](http://ianozsvald.com/2008/11/17/making-python-math-196-faster-with-shedskin/)\n\n", "quality_notes": "", "copyright_text": "Standard YouTube License", "embed": "<object width=\"640\" height=\"390\"><param name=\"movie\" value=\"http://youtube.com/v/F7NOKSP_vlQ?version=3&amp;hl=en_US\"></param><param name=\"allowFullScreen\" value=\"true\"></param><param name=\"allowscriptaccess\" value=\"always\"></param><embed src=\"http://youtube.com/v/F7NOKSP_vlQ?version=3&amp;hl=en_US\" type=\"application/x-shockwave-flash\" width=\"640\" height=\"390\" allowscriptaccess=\"always\" allowfullscreen=\"true\"></embed></object>", "thumbnail_url": "http://i.ytimg.com/vi/F7NOKSP_vlQ/hqdefault.jpg", "duration": null, "video_ogv_length": null, "video_ogv_url": null, "video_ogv_download_only": false, "video_mp4_length": null, "video_mp4_url": null, "video_mp4_download_only": false, "video_webm_length": null, "video_webm_url": null, "video_webm_download_only": false, "video_flv_length": null, "video_flv_url": null, "video_flv_download_only": false, "source_url": "http://www.youtube.com/watch?v=F7NOKSP_vlQ", "whiteboard": "", "recorded": "2011-07-18", "added": "2012-08-30T21:02:23", "updated": "2014-04-08T20:28:27.776" }