Multitask multioutput GPy Coregionalized Regression with non-Gaussian Likelihood and Laplace inference function

0 votes

I want to perform coregionalized regression in GPy, however I am using a Bernoulli likelihood and then to estimate that as a Gaussian, I use Laplace inference. The code below shows how I would usually run a single-output GP with this set up (with my custom PjkRbf kernel):

likelihood  = GPy.likelihoods.Bernoulli()
laplace_inf = GPy.inference.latent_function_inference.Laplace()
kernel      = GPy.kern.PjkRbf(X.shape[1])
m = GPy.core.GP(X, Y, kernel=kernel, likelihood=likelihood, inference_method=laplace_inf)

Now I am trying to run the same set up but as multi-output. This is something I have not been able to do.

I have tried using the GPCoregionalizedRegression class with an ICM kernel, as shown in the code below:

likelihood1  = GPy.likelihoods.Bernoulli()
likelihood2 = GPy.likelihoods.Bernoulli()
laplace_inf = GPy.inference.latent_function_inference.Laplace()
K = GPy.kern.PjkRbf(X.shape[1])
icm = GPy.util.multioutput.ICM(input_dim=2,num_outputs=2,kernel=K)
m = GPy.models.GPCoregionalizedRegression([X,X],[Y1,Y2],
                                          likelihoods_list=[likelihood1, likelihood2])

Running this code throws an AssertionError, with a long stack-trace, but the last section shows the following. The likelihood cannot be asserted as a Gaussian.

~\Anaconda3\lib\site-packages\GPy\likelihoods\ in gaussian_variance(self, Y_metadata)
     23     def gaussian_variance(self, Y_metadata):
---> 24         assert all([isinstance(l, Gaussian) for l in self.likelihoods_list])
     25         ind = Y_metadata['output_index'].flatten()
     26         variance = np.zeros(ind.size)


This is because I am not able to pass the Laplace inference to the GPCoregionalizedRegression model.

Can anyone offer advice on how I can resolve this, or if there is a different model I can use to perform multioutput regression with a Bernoulli likelihood and Laplace inference method?

Mar 26, 2022 in Machine Learning by Dev
• 6,000 points

1 answer to this question.

0 votes

The issue isn't that you can't provide Laplace inference to GPCoregionalizedRegression; it's that your GPy.likelihoods. Bernoulli chances aren't supported.
In the documents it is given that Most likelihood classes inherit directly from GPy.likelihoods.likelihood, although an intermediary class GPy.likelihoods.mixed_noise.MixedNoise is used by GPy.likelihoods.multioutput_likelihood.
The underlying likelihood is a GPy.likelihoods.mixed noise because you set up a multi-output problem. In GPy, the MixedNoise object only supports lists of GPy.likelihoods. Objects with a Gaussian distribution.
The source code for comparison:

class MixedNoise(Likelihood):
def __init__(self, likelihoods_list, name='mixed_noise'):
    #NOTE at the moment this likelihood only works for using a list of gaussians
    super(Likelihood, self).__init__(name=name)

As a result, the assert statement in \GPy\likelihoods\mixed, which tests that the likelihood_list items are Gaussian, fails, as shown by the stacktrace you provided.

As of right moment, this hasn't been implemented. I believe that the development of GPy is given less importance than that of its replacement, GPflow. There appear to be no constraints on the type of likelihoods in the appropriate functions (according to the code used in this example) (even if the example again uses only Gaussian likelihoods)

answered Apr 4, 2022 by Nandini
• 5,480 points

Related Questions In Machine Learning

0 votes
0 answers

Difference between classification and regression, with SVMs

What is the exact difference between a ...READ MORE

Feb 28, 2022 in Machine Learning by Dev
• 6,000 points
0 votes
1 answer

Can I draw a regression line and show parameters using scatterplot with a pandas dataframe?

There isn't a parameter for DataFrame.plot(). Seaborn, ...READ MORE

answered Mar 7, 2022 in Machine Learning by Dev
• 6,000 points
0 votes
2 answers
+1 vote
2 answers

how can i count the items in a list?

Syntax :            list. count(value) Code: colors = ['red', 'green', ...READ MORE

answered Jul 7, 2019 in Python by Neha
• 330 points

edited Jul 8, 2019 by Kalgi 4,133 views
0 votes
1 answer
0 votes
1 answer
0 votes
1 answer

What is the difference between linear regression and logistic regression?

Hi Dev, to answer your question Linear Regression ...READ MORE

answered Feb 2, 2022 in Machine Learning by Nandini
• 5,480 points
webinar_success Thank you for registering Join Edureka Meetup community for 100+ Free Webinars each month JOIN MEETUP GROUP