Alternatives to linear regression for dataset with many points with small value and some extreme values

Question

I want to model the pharmaceutical costs for a group of patients for the next year, based on this years pharma data (codes for the pharmaceuticals), age, gender and this year's costs.

I used linear regression and got an R^2 of 0.69, which was surprisingly good. When I devided the patients into 5 groups of the same size based on the amount of costs for the current year, I could see that the bottom 80 % performed extremely poorly, while the top 20% made up for it with a score of 0.71.

80% of the people have costs roughly under 500 Euro, while those who have a lot of costs, have extreme costs, up to 500.000 Euros.

I think, since Linear Regression wants to minimise residuals, predicting the bottom costs with the still relatively small residuals does not bring as much gain as minimising high costs.

Is there an alternative model, that would be more useful in this context to predict small costs better as well?

Answer 1

The above situation is the case where variance increased with the expected mean this is heteroscedasticity.
You can choose from the following:

• Use WLS and use weights depending on the predicted value 
•  Convert the dependent variable. Example: log (y) and  lognormal model estimation 
•  Use a distribution where the variance increases on average. 

 For instance The Poisson  variance is equal to the mean. You must use quasi-Poisson for continuous variables. Gamma has a squared variance on average. These distributions are typically implemented in GLM.