http://www.scipy-lectures.org/index.html Navigation * next * Scipy lecture notes >> * Collapse to compact view Collapse document to compact view * Edit Improve this page: Edit it on Github. Scipy Lecture NotesP One document to learn numerics, science, and data with PythonP Download PDF, 2 pages per side PDF, 1 page per side HTML and example files Source code (github) Tutorials on the scientific Python ecosystem: a quick introduction to central tools and techniques. The different chapters each correspond to a 1 to 2 hours course with increasing level of expertise, from beginner to expert. Release: 2020.2 * About the scipy lecture notes + Authors + What's new + License + Contributing * 1. Getting started with Python for science + 1.1. Python scientific computing ecosystem o 1.1.1. Why Python? # 1.1.1.1. The scientist's needs # 1.1.1.2. Python's strengths # 1.1.1.3. How does Python compare to other solutions? @ Compiled languages: C, C++, Fortran... @ Matlab scripting language @ Julia @ Other scripting languages: Scilab, Octave, R, IDL, etc. @ Python o 1.1.2. The Scientific Python ecosystem o 1.1.3. Before starting: Installing a working environment o 1.1.4. The workflow: interactive environments and text editors # 1.1.4.1. Interactive work # 1.1.4.2. Elaboration of the work in an editor # 1.1.4.3. IPython and Jupyter Tips and Tricks + 1.2. The Python language o 1.2.1. First steps o 1.2.2. Basic types # 1.2.2.1. Numerical types # 1.2.2.2. Containers @ Lists @ Strings @ Dictionaries @ More container types # 1.2.2.3. Assignment operator o 1.2.3. Control Flow # 1.2.3.1. if/elif/else # 1.2.3.2. for/range # 1.2.3.3. while/break/continue # 1.2.3.4. Conditional Expressions # 1.2.3.5. Advanced iteration @ Iterate over any sequence @ Keeping track of enumeration number @ Looping over a dictionary # 1.2.3.6. List Comprehensions o 1.2.4. Defining functions # 1.2.4.1. Function definition # 1.2.4.2. Return statement # 1.2.4.3. Parameters # 1.2.4.4. Passing by value # 1.2.4.5. Global variables # 1.2.4.6. Variable number of parameters # 1.2.4.7. Docstrings # 1.2.4.8. Functions are objects # 1.2.4.9. Methods # 1.2.4.10. Exercises o 1.2.5. Reusing code: scripts and modules # 1.2.5.1. Scripts # 1.2.5.2. Importing objects from modules # 1.2.5.3. Creating modules # 1.2.5.4. '__main__' and module loading # 1.2.5.5. Scripts or modules? How to organize your code @ How modules are found and imported # 1.2.5.6. Packages # 1.2.5.7. Good practices o 1.2.6. Input and Output # 1.2.6.1. Iterating over a file @ File modes o 1.2.7. Standard Library # 1.2.7.1. os module: operating system functionality @ Directory and file manipulation @ os.path: path manipulations @ Running an external command @ Walking a directory @ Environment variables: # 1.2.7.2. shutil: high-level file operations # 1.2.7.3. glob: Pattern matching on files # 1.2.7.4. sys module: system-specific information # 1.2.7.5. pickle: easy persistence o 1.2.8. Exception handling in Python # 1.2.8.1. Exceptions # 1.2.8.2. Catching exceptions @ try/except @ try/finally @ Easier to ask for forgiveness than for permission # 1.2.8.3. Raising exceptions o 1.2.9. Object-oriented programming (OOP) + 1.3. Python 2 and Python 3 o 1.3.1. A very short summary o 1.3.2. Breaking changes between Python 2 and Python 3 # 1.3.2.1. Print function # 1.3.2.2. Division o 1.3.3. Some new features in Python 3 + 1.4. NumPy: creating and manipulating numerical data o 1.4.1. The NumPy array object # 1.4.1.1. What are NumPy and NumPy arrays? @ NumPy arrays @ NumPy Reference documentation @ Import conventions # 1.4.1.2. Creating arrays @ Manual construction of arrays @ Functions for creating arrays # 1.4.1.3. Basic data types # 1.4.1.4. Basic visualization # 1.4.1.5. Indexing and slicing # 1.4.1.6. Copies and views # 1.4.1.7. Fancy indexing @ Using boolean masks @ Indexing with an array of integers o 1.4.2. Numerical operations on arrays # 1.4.2.1. Elementwise operations @ Basic operations @ Other operations # 1.4.2.2. Basic reductions @ Computing sums @ Other reductions # 1.4.2.3. Broadcasting # 1.4.2.4. Array shape manipulation @ Flattening @ Reshaping @ Adding a dimension @ Dimension shuffling @ Resizing # 1.4.2.5. Sorting data # 1.4.2.6. Summary o 1.4.3. More elaborate arrays # 1.4.3.1. More data types @ Casting @ Different data type sizes # 1.4.3.2. Structured data types # 1.4.3.3. maskedarray: dealing with (propagation of) missing data o 1.4.4. Advanced operations # 1.4.4.1. Polynomials @ More polynomials (with more bases) # 1.4.4.2. Loading data files @ Text files @ Images @ NumPy's own format @ Well-known (& more obscure) file formats o 1.4.5. Some exercises # 1.4.5.1. Array manipulations # 1.4.5.2. Picture manipulation: Framing a Face # 1.4.5.3. Data statistics # 1.4.5.4. Crude integral approximations # 1.4.5.5. Mandelbrot set # 1.4.5.6. Markov chain o 1.4.6. Full code examples # 1.4.6.1. Full code examples for the numpy chapter + 1.5. Matplotlib: plotting o 1.5.1. Introduction # 1.5.1.1. IPython, Jupyter, and matplotlib modes # 1.5.1.2. pyplot o 1.5.2. Simple plot # 1.5.2.1. Plotting with default settings # 1.5.2.2. Instantiating defaults # 1.5.2.3. Changing colors and line widths # 1.5.2.4. Setting limits # 1.5.2.5. Setting ticks # 1.5.2.6. Setting tick labels # 1.5.2.7. Moving spines # 1.5.2.8. Adding a legend # 1.5.2.9. Annotate some points # 1.5.2.10. Devil is in the details o 1.5.3. Figures, Subplots, Axes and Ticks # 1.5.3.1. Figures # 1.5.3.2. Subplots # 1.5.3.3. Axes # 1.5.3.4. Ticks @ Tick Locators o 1.5.4. Other Types of Plots: examples and exercises # 1.5.4.1. Regular Plots # 1.5.4.2. Scatter Plots # 1.5.4.3. Bar Plots # 1.5.4.4. Contour Plots # 1.5.4.5. Imshow # 1.5.4.6. Pie Charts # 1.5.4.7. Quiver Plots # 1.5.4.8. Grids # 1.5.4.9. Multi Plots # 1.5.4.10. Polar Axis # 1.5.4.11. 3D Plots # 1.5.4.12. Text o 1.5.5. Beyond this tutorial # 1.5.5.1. Tutorials # 1.5.5.2. Matplotlib documentation # 1.5.5.3. Code documentation # 1.5.5.4. Galleries # 1.5.5.5. Mailing lists o 1.5.6. Quick references # 1.5.6.1. Line properties # 1.5.6.2. Line styles # 1.5.6.3. Markers # 1.5.6.4. Colormaps o 1.5.7. Full code examples # 1.5.7.1. Code samples for Matplotlib # 1.5.7.2. Code for the chapter's exercises # 1.5.7.3. Example demoing choices for an option # 1.5.7.4. Code generating the summary figures with a title + 1.6. Scipy : high-level scientific computing o 1.6.1. File input/output: scipy.io o 1.6.2. Special functions: scipy.special o 1.6.3. Linear algebra operations: scipy.linalg o 1.6.4. Interpolation: scipy.interpolate o 1.6.5. Optimization and fit: scipy.optimize # 1.6.5.1. Curve fitting # 1.6.5.2. Finding the minimum of a scalar function # 1.6.5.3. Finding the roots of a scalar function o 1.6.6. Statistics and random numbers: scipy.stats # 1.6.6.1. Distributions: histogram and probability density function # 1.6.6.2. Mean, median and percentiles # 1.6.6.3. Statistical tests o 1.6.7. Numerical integration: scipy.integrate # 1.6.7.1. Function integrals # 1.6.7.2. Integrating differential equations o 1.6.8. Fast Fourier transforms: scipy.fftpack o 1.6.9. Signal processing: scipy.signal o 1.6.10. Image manipulation: scipy.ndimage # 1.6.10.1. Geometrical transformations on images # 1.6.10.2. Image filtering # 1.6.10.3. Mathematical morphology # 1.6.10.4. Connected components and measurements on images o 1.6.11. Summary exercises on scientific computing # 1.6.11.1. Maximum wind speed prediction at the Sprogo station @ Statistical approach @ Computing the cumulative probabilities @ Prediction with UnivariateSpline @ Exercise with the Gumbell distribution # 1.6.11.2. Non linear least squares curve fitting: application to point extraction in topographical lidar data @ Introduction @ Loading and visualization @ Fitting a waveform with a simple Gaussian model - Model - Initial solution - Fit @ Going further # 1.6.11.3. Image processing application: counting bubbles and unmolten grains @ Statement of the problem # 1.6.11.4. Example of solution for the image processing exercise: unmolten grains in glass o 1.6.12. Full code examples for the scipy chapter # 1.6.12.18. Solutions of the exercises for scipy + 1.7. Getting help and finding documentation * 2. Advanced topics + 2.1. Advanced Python Constructs o 2.1.1. Iterators, generator expressions and generators # 2.1.1.1. Iterators # 2.1.1.2. Generator expressions # 2.1.1.3. Generators # 2.1.1.4. Bidirectional communication # 2.1.1.5. Chaining generators o 2.1.2. Decorators # 2.1.2.1. Replacing or tweaking the original object # 2.1.2.2. Decorators implemented as classes and as functions # 2.1.2.3. Copying the docstring and other attributes of the original function # 2.1.2.4. Examples in the standard library # 2.1.2.5. Deprecation of functions # 2.1.2.6. A while-loop removing decorator # 2.1.2.7. A plugin registration system o 2.1.3. Context managers # 2.1.3.1. Catching exceptions # 2.1.3.2. Using generators to define context managers + 2.2. Advanced NumPy o 2.2.1. Life of ndarray # 2.2.1.1. It's... # 2.2.1.2. Block of memory # 2.2.1.3. Data types @ The descriptor @ Example: reading .wav files @ Casting and re-interpretation/views - Casting - Re-interpretation / viewing # 2.2.1.4. Indexing scheme: strides @ Main point - C and Fortran order - Slicing with integers @ Example: fake dimensions with strides @ Broadcasting @ More tricks: diagonals @ CPU cache effects # 2.2.1.5. Findings in dissection o 2.2.2. Universal functions # 2.2.2.1. What they are? @ Parts of an Ufunc @ Making it easier # 2.2.2.2. Exercise: building an ufunc from scratch # 2.2.2.3. Solution: building an ufunc from scratch # 2.2.2.4. Generalized ufuncs o 2.2.3. Interoperability features # 2.2.3.1. Sharing multidimensional, typed data # 2.2.3.2. The old buffer protocol # 2.2.3.3. The old buffer protocol # 2.2.3.4. Array interface protocol o 2.2.4. Array siblings: chararray, maskedarray, matrix # 2.2.4.1. chararray: vectorized string operations # 2.2.4.2. masked_array missing data @ The mask @ Domain-aware functions # 2.2.4.3. recarray: purely convenience # 2.2.4.4. matrix: convenience? o 2.2.5. Summary o 2.2.6. Contributing to NumPy/Scipy # 2.2.6.1. Why # 2.2.6.2. Reporting bugs @ Good bug report # 2.2.6.3. Contributing to documentation # 2.2.6.4. Contributing features # 2.2.6.5. How to help, in general + 2.3. Debugging code o 2.3.1. Avoiding bugs # 2.3.1.1. Coding best practices to avoid getting in trouble # 2.3.1.2. pyflakes: fast static analysis @ Running pyflakes on the current edited file @ A type-as-go spell-checker like integration o 2.3.2. Debugging workflow o 2.3.3. Using the Python debugger # 2.3.3.1. Invoking the debugger @ Postmortem @ Step-by-step execution @ Other ways of starting a debugger # 2.3.3.2. Debugger commands and interaction @ Getting help when in the debugger o 2.3.4. Debugging segmentation faults using gdb + 2.4. Optimizing code o 2.4.1. Optimization workflow o 2.4.2. Profiling Python code # 2.4.2.1. Timeit # 2.4.2.2. Profiler # 2.4.2.3. Line-profiler o 2.4.3. Making code go faster # 2.4.3.1. Algorithmic optimization @ Example of the SVD o 2.4.4. Writing faster numerical code # 2.4.4.1. Additional Links + 2.5. Sparse Matrices in SciPy o 2.5.1. Introduction # 2.5.1.1. Why Sparse Matrices? # 2.5.1.2. Sparse Matrices vs. Sparse Matrix Storage Schemes # 2.5.1.3. Typical Applications # 2.5.1.4. Prerequisites # 2.5.1.5. Sparsity Structure Visualization o 2.5.2. Storage Schemes # 2.5.2.1. Common Methods # 2.5.2.2. Sparse Matrix Classes @ Diagonal Format (DIA) - Examples @ List of Lists Format (LIL) - Examples @ Dictionary of Keys Format (DOK) - Examples @ Coordinate Format (COO) - Examples @ Compressed Sparse Row Format (CSR) - Examples @ Compressed Sparse Column Format (CSC) - Examples @ Block Compressed Row Format (BSR) - Examples # 2.5.2.3. Summary o 2.5.3. Linear System Solvers # 2.5.3.1. Sparse Direct Solvers @ Examples # 2.5.3.2. Iterative Solvers @ Common Parameters @ LinearOperator Class @ A Few Notes on Preconditioning # 2.5.3.3. Eigenvalue Problem Solvers @ The eigen module o 2.5.4. Other Interesting Packages + 2.6. Image manipulation and processing using Numpy and Scipy o 2.6.1. Opening and writing to image files o 2.6.2. Displaying images o 2.6.3. Basic manipulations # 2.6.3.1. Statistical information # 2.6.3.2. Geometrical transformations o 2.6.4. Image filtering # 2.6.4.1. Blurring/smoothing # 2.6.4.2. Sharpening # 2.6.4.3. Denoising # 2.6.4.4. Mathematical morphology o 2.6.5. Feature extraction # 2.6.5.1. Edge detection # 2.6.5.2. Segmentation o 2.6.6. Measuring objects properties: ndimage.measurements o 2.6.7. Full code examples o 2.6.8. Examples for the image processing chapter + 2.7. Mathematical optimization: finding minima of functions o 2.7.1. Knowing your problem # 2.7.1.1. Convex versus non-convex optimization # 2.7.1.2. Smooth and non-smooth problems # 2.7.1.3. Noisy versus exact cost functions # 2.7.1.4. Constraints o 2.7.2. A review of the different optimizers # 2.7.2.1. Getting started: 1D optimization # 2.7.2.2. Gradient based methods @ Some intuitions about gradient descent @ Conjugate gradient descent # 2.7.2.3. Newton and quasi-newton methods @ Newton methods: using the Hessian (2nd differential) @ Quasi-Newton methods: approximating the Hessian on the fly o 2.7.3. Full code examples o 2.7.4. Examples for the mathematical optimization chapter # 2.7.4.12. Gradient-less methods @ A shooting method: the Powell algorithm @ Simplex method: the Nelder-Mead # 2.7.4.13. Global optimizers @ Brute force: a grid search o 2.7.5. Practical guide to optimization with scipy # 2.7.5.1. Choosing a method # 2.7.5.2. Making your optimizer faster # 2.7.5.3. Computing gradients # 2.7.5.4. Synthetic exercices o 2.7.6. Special case: non-linear least-squares # 2.7.6.1. Minimizing the norm of a vector function # 2.7.6.2. Curve fitting o 2.7.7. Optimization with constraints # 2.7.7.1. Box bounds # 2.7.7.2. General constraints o 2.7.8. Full code examples o 2.7.9. Examples for the mathematical optimization chapter + 2.8. Interfacing with C o 2.8.1. Introduction o 2.8.2. Python-C-Api # 2.8.2.1. Example # 2.8.2.2. Numpy Support o 2.8.3. Ctypes # 2.8.3.1. Example # 2.8.3.2. Numpy Support o 2.8.4. SWIG # 2.8.4.1. Example # 2.8.4.2. Numpy Support o 2.8.5. Cython # 2.8.5.1. Example # 2.8.5.2. Numpy Support o 2.8.6. Summary o 2.8.7. Further Reading and References o 2.8.8. Exercises # 2.8.8.1. Python-C-API # 2.8.8.2. Ctypes # 2.8.8.3. SWIG # 2.8.8.4. Cython * 3. Packages and applications + 3.1. Statistics in Python o 3.1.1. Data representation and interaction # 3.1.1.1. Data as a table # 3.1.1.2. The pandas data-frame @ Creating dataframes: reading data files or converting arrays @ Manipulating data @ Plotting data o 3.1.2. Hypothesis testing: comparing two groups # 3.1.2.1. Student's t-test: the simplest statistical test @ 1-sample t-test: testing the value of a population mean @ 2-sample t-test: testing for difference across populations # 3.1.2.2. Paired tests: repeated measurements on the same individuals o 3.1.3. Linear models, multiple factors, and analysis of variance # 3.1.3.1. "formulas" to specify statistical models in Python @ A simple linear regression @ Categorical variables: comparing groups or multiple categories # 3.1.3.2. Multiple Regression: including multiple factors # 3.1.3.3. Post-hoc hypothesis testing: analysis of variance (ANOVA) o 3.1.4. More visualization: seaborn for statistical exploration # 3.1.4.1. Pairplot: scatter matrices # 3.1.4.2. lmplot: plotting a univariate regression o 3.1.5. Testing for interactions o 3.1.6. Full code for the figures o 3.1.7. Solutions to this chapter's exercises + 3.2. Sympy : Symbolic Mathematics in Python o 3.2.1. First Steps with SymPy # 3.2.1.1. Using SymPy as a calculator # 3.2.1.2. Symbols o 3.2.2. Algebraic manipulations # 3.2.2.1. Expand # 3.2.2.2. Simplify o 3.2.3. Calculus # 3.2.3.1. Limits # 3.2.3.2. Differentiation # 3.2.3.3. Series expansion # 3.2.3.4. Integration o 3.2.4. Equation solving o 3.2.5. Linear Algebra # 3.2.5.1. Matrices # 3.2.5.2. Differential Equations + 3.3. Scikit-image: image processing o 3.3.1. Introduction and concepts # 3.3.1.1. scikit-image and the SciPy ecosystem # 3.3.1.2. What's to be found in scikit-image o 3.3.2. Input/output, data types and colorspaces # 3.3.2.1. Data types # 3.3.2.2. Colorspaces o 3.3.3. Image preprocessing / enhancement # 3.3.3.1. Local filters # 3.3.3.2. Non-local filters # 3.3.3.3. Mathematical morphology o 3.3.4. Image segmentation # 3.3.4.1. Binary segmentation: foreground + background @ Histogram-based method: Otsu thresholding @ Labeling connected components of a discrete image # 3.3.4.2. Marker based methods @ Watershed segmentation @ Random walker segmentation o 3.3.5. Measuring regions' properties o 3.3.6. Data visualization and interaction o 3.3.7. Feature extraction for computer vision o 3.3.8. Full code examples o 3.3.9. Examples for the scikit-image chapter + 3.4. Traits: building interactive dialogs o 3.4.1. Introduction o 3.4.2. Example o 3.4.3. What are Traits # 3.4.3.1. Initialisation # 3.4.3.2. Validation # 3.4.3.3. Documentation # 3.4.3.4. Visualization: opening a dialog # 3.4.3.5. Deferral # 3.4.3.6. Notification # 3.4.3.7. Some more advanced traits + 3.5. 3D plotting with Mayavi o 3.5.1. Mlab: the scripting interface # 3.5.1.1. 3D plotting functions @ Points @ Lines @ Elevation surface @ Arbitrary regular mesh @ Volumetric data # 3.5.1.2. Figures and decorations @ Figure management @ Changing plot properties @ Decorations o 3.5.2. Interactive work # 3.5.2.1. The "pipeline dialog" # 3.5.2.2. The script recording button o 3.5.3. Slicing and dicing data: sources, modules and filters # 3.5.3.1. An example: inspecting magnetic fields # 3.5.3.2. Different views on data: sources and modules @ Different sources: scatters and fields @ Transforming data: filters @ mlab.pipeline: the scripting layer o 3.5.4. Animating the data o 3.5.5. Making interactive dialogs # 3.5.5.1. A simple dialog # 3.5.5.2. Making it interactive o 3.5.6. Putting it together + 3.6. scikit-learn: machine learning in Python o 3.6.1. Introduction: problem settings # 3.6.1.1. What is machine learning? # 3.6.1.2. Data in scikit-learn @ The data matrix @ A Simple Example: the Iris Dataset - The application problem - Loading the Iris Data with Scikit-learn o 3.6.2. Basic principles of machine learning with scikit-learn # 3.6.2.1. Introducing the scikit-learn estimator object @ Fitting on data # 3.6.2.2. Supervised Learning: Classification and regression # 3.6.2.3. A recap on Scikit-learn's estimator interface # 3.6.2.4. Regularization: what it is and why it is necessary @ Prefering simpler models @ Simple versus complex models for classification o 3.6.3. Supervised Learning: Classification of Handwritten Digits # 3.6.3.1. The nature of the data # 3.6.3.2. Visualizing the Data on its principal components # 3.6.3.3. Gaussian Naive Bayes Classification # 3.6.3.4. Quantitative Measurement of Performance o 3.6.4. Supervised Learning: Regression of Housing Data # 3.6.4.1. A quick look at the data # 3.6.4.2. Predicting Home Prices: a Simple Linear Regression o 3.6.5. Measuring prediction performance # 3.6.5.1. A quick test on the K-neighbors classifier # 3.6.5.2. A correct approach: Using a validation set # 3.6.5.3. Model Selection via Validation # 3.6.5.4. Cross-validation # 3.6.5.5. Hyperparameter optimization with cross-validation @ Basic Hyperparameter Optimization @ Automatically Performing Grid Search @ Built-in Hyperparameter Search @ Nested cross-validation o 3.6.6. Unsupervised Learning: Dimensionality Reduction and Visualization # 3.6.6.1. Dimensionality Reduction: PCA # 3.6.6.2. Visualization with a non-linear embedding: tSNE o 3.6.7. The eigenfaces example: chaining PCA and SVMs o 3.6.8. The eigenfaces example: chaining PCA and SVMs # 3.6.8.1. Preprocessing: Principal Component Analysis # 3.6.8.2. Doing the Learning: Support Vector Machines # 3.6.8.3. Pipelining o 3.6.9. Parameter selection, Validation, and Testing # 3.6.9.1. Hyperparameters, Over-fitting, and Under-fitting @ Bias-variance trade-off: illustration on a simple regression problem # 3.6.9.2. Visualizing the Bias/Variance Tradeoff @ Validation Curves @ Learning Curves # 3.6.9.3. Summary on model selection @ High Bias @ High Variance # 3.6.9.4. A last word of caution: separate validation and test set o 3.6.10. Examples for the scikit-learn chapter ScipyLectures.pdf ScipyLectures-simple.pdf _images/plusBox.png _images/logo.svg Table Of Contents * Scipy Lecture Notes + One document to learn numerics, science, and data with Python Next topic About the scipy lecture notes This Page * Show Source Quick search [ ] [Go] Navigation * next * Scipy lecture notes >> * Collapse to compact view Collapse document to compact view * Edit Improve this page: Edit it on Github. (c) Copyright 2012,2013,2015,2016,2017,2018,2019,2020. Created using Sphinx 1.7.9.