Parameter learning with Pandas

This notebook uses pandas to learn the parameters.

However the simplest way to learn parameters is to use ``BNLearner`` :-).

Moreover, you will be able to add priors, etc (see learning BN).

%matplotlib inline
from pylab import *
import matplotlib.pyplot as plt

import os

Importing pyAgrum

import pyAgrum as gum
import pyAgrum.lib.notebook as gnb

Loading two BNs

bn=gum.loadBN("res/asia.bif")
bn2=gum.loadBN("res/asia.bif")

gnb.sideBySide(bn,bn2,
               captions=['First bn','Second bn'])

First bn

Second bn

Randomizing the parameters

bn.generateCPTs()
bn2.generateCPTs()

Direct comparison of parameters

from IPython.display import HTML

gnb.sideBySide(bn.cpt(3),
               bn2.cpt(3),
               captions=['<h3>cpt of node 3 in first bn</h3>','<h3>same cpt in second bn</h3>'])

positive_XraY tuberculos_or_cancer 0 1 0 0.7696 0.2304 1 0.5382 0.4618 cpt of node 3 in first bn

positive_XraY tuberculos_or_cancer 0 1 0 0.2009 0.7991 1 0.4566 0.5434 same cpt in second bn

Exact KL-divergence

Since the BN is not too big, BruteForceKL can be computed …

g1=gum.ExactBNdistance(bn,bn2)
before_learning=g1.compute()
print(f"klPQ computed : {before_learning['klPQ']}")

klPQ computed : 2.8668227849210015

Just to be sure that the distance between a BN and itself is 0 :

g0=gum.ExactBNdistance(bn,bn)
print(g0.compute()['klPQ'])

0.0

Generate a database from the original BN

gum.generateSample(bn,10000,"out/test.csv",True)

out/test.csv: 100%|████████████████████████████████████████|

Log2-Likelihood : -69671.19528080855

-69671.19528080855

Using pandas for _counting

As an exercise, we will use pandas to learn the parameters.

# using bn as a template for the specification of variables in test.csv
learner=gum.BNLearner("out/test.csv",bn)
bn3=learner.learnParameters(bn.dag())

#the same but we add a Laplace adjustment (smoothing) as a Prior
learner=gum.BNLearner("out/test.csv",bn)
learner.useSmoothingPrior(1000) # a count C is replaced by C+1000
bn4=learner.learnParameters(bn.dag())

after_pyAgrum_learning=gum.ExactBNdistance(bn,bn3).compute()
after_pyAgrum_learning_with_smoothing=gum.ExactBNdistance(bn,bn4).compute()
print("without prior:{}".format(after_pyAgrum_learning['klPQ']))
print("with prior smooting(1000):{}".format(after_pyAgrum_learning_with_smoothing['klPQ']))

without prior:0.0018716290061952438
with prior smooting(1000):0.08948433938058356

Now, let’s try to learn the parameters with pandas

import pandas

# We directly generate samples in a DataFrame
df,_=gum.generateSample(bn,10000,None,True)

100%|██████████████████████████████████████████████████████|

Log2-Likelihood : -70285.97132541901

df.head()

	positive_XraY	bronchitis	dyspnoea	tuberculosis	visit_to_Asia	tuberculos_or_cancer	smoking	lung_cancer
0	0	0	1	1	0	0	0	1
1	1	1	0	0	0	1	1	1
2	0	1	1	1	0	0	0	1
3	0	1	0	1	0	0	1	1
4	1	1	1	0	0	0	0	1

We use the crosstab function in pandas

c=pandas.crosstab(df['dyspnoea'],[df['tuberculos_or_cancer'],df['bronchitis']])
c

tuberculos_or_cancer	0		1
bronchitis	0	1	0	1
dyspnoea
0	1130	1610	747	993
1	1105	1538	1160	1717

Playing with numpy reshaping, we retrieve the good form for the CPT from the pandas cross-table

gnb.sideBySide('<pre>'+str(np.array((c/c.sum().apply(np.float32)).transpose()).reshape(2,2,2))+'</pre>',
               bn.cpt('dyspnoea'),
               captions=["<h3>Learned parameters in crosstab","<h3>Original parameters in bn</h3>"])

[[[0.50559284 0.49440716] [0.51143583 0.48856417]] [[0.39171474 0.60828526] [0.36642066 0.63357934]]] Learned parameters in crosstab

dyspnoea bronchitis tuberculos_or_cancer 0 1 0 0 0.5101 0.4899 1 0.3689 0.6311 1 0 0.5113 0.4887 1 0.3836 0.6164 Original parameters in bn

A global method for estimating Bayesian network parameters from CSV file using PANDAS

def computeCPTfromDF(bn,df,name):
    """
    Compute the CPT of variable "name" in the BN bn from the database df
    """
    id=bn.idFromName(name)
    parents=list(reversed(bn.cpt(id).names))
    domains=[bn[name].domainSize()
             for name in parents]
    parents.pop()

    if (len(parents)>0):
        c=pandas.crosstab(df[name],[df[parent] for parent in parents])
        s=c/c.sum().apply(np.float32)
    else:
        s=df[name].value_counts(normalize=True)

    bn.cpt(id)[:]=np.array((s).transpose()).reshape(*domains)

def ParametersLearning(bn,df):
    """
    Compute the CPTs of every varaible in the BN bn from the database df
    """
    for name in bn.names():
        computeCPTfromDF(bn,df,name)

ParametersLearning(bn2,df)

KL has decreased a lot (if everything’s OK)

g1=gum.ExactBNdistance(bn,bn2)
print("BEFORE LEARNING")
print(before_learning['klPQ'])
print
print("AFTER LEARNING")
print(g1.compute()['klPQ'])

BEFORE LEARNING
2.8668227849210015
AFTER LEARNING
0.33231734269251345

And CPTs should be close

gnb.sideBySide(bn.cpt(3),
               bn2.cpt(3),
               captions=["<h3>Original BN","<h3>learned BN</h3>"])

positive_XraY tuberculos_or_cancer 0 1 0 0.7696 0.2304 1 0.5382 0.4618 Original BN

positive_XraY tuberculos_or_cancer 0 1 0 0.7698 0.2302 1 0.5337 0.4663 learned BN

Influence of the size of the database on the quality of learned parameters

What is the effect of increasing the size of the database on the KL ? We expect that the KL decreases to 0.

res=[]
for i in range(200,10001,50):
    ParametersLearning(bn2,df[:i])
    g1=gum.ExactBNdistance(bn,bn2)
    res.append(g1.compute()['klPQ'])
fig=figure(figsize=(10,6))
ax  = fig.add_subplot(1, 1, 1)
ax.plot(range(200,10001,50),res)
ax.set_xlabel("size of the database")
ax.set_ylabel("KL")
ax.set_title("klPQ(bn,learnedBN(x))");