Increase the cell width of the Jupyter notebook in browser

Sometimes you may need to use the full-screen size for showing animation, pictures or embed iframe web in your work. By default, the notebook only uses a small part of it.  Assume you only want to change current notebook but not the default setting, here is how:

from IPython.core.display import display, HTML

display(HTML(“<style>.container {width:100% !important;}</style>”))

Alternatively, you can also set the CSS of a notebook by reading from a pre-defined css file.

First, create a file containing the css setting:


If the file name is custom_ipython.css then add a cell containing:

from IPython.core.display import HTML
def css_styling():
styles = open(“custom.css”, “r”).read()
return HTML(styles)


Numerical instability in deep learning with softmax

One of the most frequently used activation function in output layers for multi-class classification neural network is softmax. Softmax is defined as f(X) = exp(xi)/sum(exp(xi)) and it returns probability for each individual class with all probability the sum of one. For the two-class problem, sigmoid will return the same probability as softmax.

While translating softmax into program code, there are some little thing to watch out due to numerical instability associated with exploding gradient or vanishing gradient.  Let’s look at two examples:

Screen Shot 2017-10-09 at 9.07.14 PM

While the weights increase 1000 X, the probability becomes useless, either 0 or 1.

Screen Shot 2017-10-09 at 9.16.37 PM

The same thing happens to the weights vanishing. All the weights now share the same probability as the weights approach zero.

However, there is an easy fix for addressing the exploding and vanishing weights by modifying the softmax function to softmax(X + c). Mostly commonly used C is max(X) and it leaves the weight vector to be all negative. It will rule out overflow and vanishing denominator with at least one zero elements. Underflow some but not all weights are harmless.

Let’s see the impact.

Screen Shot 2017-10-09 at 9.33.02 PM

Screen Shot 2017-10-09 at 9.30.31 PM

Screen Shot 2017-10-09 at 9.30.40 PM

As seen from the two examples with stable syntax, it saved more weights from vanishing or exploding gradient.

Display all the data column in Jupyter Notebook

During our data exploration, there are often times that too many data columns in the dataframe. By default, the Jupyter Notebook only display a handful of it for simplicity.

Here is the couple of ways to display all the columns:

import pandas as pd

from IPython.display import display

data = pd.read_csv(‘mydave.cvs’)

# Direclty set the options

pd.options.display.max_columns = None


Or, you set_option method from pandas.

pd.set_option(‘display.max_columns’, None)


To locally change the setting for an only specific cell, do the following:

with pd.option_context(‘display.max_columns’,None):


You can also do:

from IPython.display import HTML


Using R in Jupyter Notebook

R has started to gain momentum in data science due to its easy-to-use and full of statistic packages. For longtime Python user, I want to run some R commands within Jupyter for pratical reasons, like some collaborators are using R for some tasks or just convenience. This article will show you how to do it.

  • Setup environment

Install R essentials in your current environment:

conda install -c r r-essentials

These ‘essentials’ include the packages dplyr, shiny, ggplot2, tidyr, caret and nnet. 

You can also create a new environment just for the R essentials:

conda create -n my-r-env -c r r-essentials

Now you’re all set to work with R in Jupyter.

How about install new packages in R for my usage in Jupyter?

There are two ways of doing it: 1. build a Conda R package by running:

conda skeleton cran xxx conda build r-xxx/

Or you can install the package from inside of R via install.packages() or devtools::install_github. But with one change: change the destination to conda R library.


  • Into good hands

THe interactivity comes mainly from the so-called “magic commands” which allows you to switch from Python to command line instructions (like ls, cat etc) or to write code in other languages such as R, Scala, Julia, …

After open Jupiter notebook, you should be able to see R in the console:


To switch from Python to R, first download the following pacakge:

%load_ext rpy2.ipython

After that, start to use R with the %R magic command.

# Hide warnings if there are any
import warnings
# Load in the r magic
%load_ext rpy2.ipython
# We need ggplot2
%R require(ggplot2)
# Load in the pandas library
import pandas as pd
# Make a pandas DataFrame
df = pd.DataFrame({‘Alphabet’: [‘a’, ‘b’, ‘c’, ‘d’,’e’, ‘f’, ‘g’, ‘h’,’i’],
‘A’: [4, 3, 5, 2, 1, 7, 7, 5, 9],
‘B’: [0, 4, 3, 6, 7, 10,11, 9, 13],
‘C’: [1, 2, 3, 1, 2, 3, 1, 2, 3]})
# Take the name of input variable df and assign it to an R variable of the same name
%%R -i df
# Plot the DataFrame df
ggplot(data=df) + geom_point(aes(x=A, y=B, color=C))

Python traps you should know

Like every language, there are some easy to overlook traps when writing Python programs. Some of the traps are hidden and can cause big problems or errors for your program. Here are some of the most common traps a good Python programmer should be aware:

    •  1. a mutable object used as the default parameter

Like all the other languages, Python provides default parameters for functions which are great for making thing easier. However, things can become unpleasant if you have put a mutable object in the function as the default value for a parameter. Let’s look at an example:


A surprise? ! The root cause is that everything is an object in Python, even function and default parameter is just an attribute of the function. Default parameter values are evaluated when the function definition is executed.

Another more obvious example:


How to fix it?

According to Python document: A way around this is to use None as the default, and explicitly test for it in the body of the function.


  • 2.  x += y vs x = x+y

Generally speaking, the two are equivalent. Let’s look at the example:


As we can see, when using +=, it returns the same id.  In the first example (53,54), x points to a new object while the latter one (55,56) it modifies its value at the current location.

  • 3. Majic parathesis ()

In Python, () can represent a tuple data structure which is immutable.


What if you only have one element in the tuple:


Majic, it becomes an integer instead of a tuple. The right thing to do is this:


  • 4. Generated element is a list of list

This is like a 2-D array. Or generating a list with mutable element in it. Sounds very easy:


By adding a value 10 into the first element in the list, we populated all the elements with the same value. Interesting, hmmm? That’s not what I want!!

The reason is still the same: mutable object within the list and they all point to the same object. The right syntax is:

As seen above, there are many traps while using Python and definitely you should be aware of it.