16402/what-are-the-ways-of-detecting-outliners-in-python

I want to set the outlier values as 'NaN' values. Here is the code I am using right now. Can someone explain me ?

import numpy as np, matplotlib.pyplot as plt data = np.random.rand(1000)+5.0 plt.plot(data) plt.xlabel('observation number') plt.ylabel('recorded value') plt.show()

Here's an implementation for the N-dimensional case (from some code for a paper here: https://github.com/joferkington/oost_paper_code/blob/master/utilities.py):

There are a huge number of ways to test for outliers, and you should give some thought to how you classify them. Ideally, you should use a-priori information (e.g. "anything above/below this value is unrealistic because...")

code from http://eurekastatistics.com/using-the-median-absolute-deviation-to-find-outliers This uses the L1 distance instead of L2 distance, and has support for asymmetric distributions.

def doubleMADsfromMedian(y,thresh=3.5): # warning: this function does not check for NAs # nor does it address issues when # more than 50% of your data have identical values m = np.median(y) abs_dev = np.abs(y - m) left_mad = np.median(abs_dev[y <= m]) right_mad = np.median(abs_dev[y >= m]) y_mad = left_mad * np.ones(len(y)) y_mad[y > m] = right_mad modified_z_score = 0.6745 * abs_dev / y_mad modified_z_score[y == m] = 0 return modified_z_score > thresh

Python provides an array of built-in constants, ...READ MORE

Sorted() sorts any sequence (list, tuple) and ...READ MORE

I have a string in Python. I ...READ MORE

Polymorphism is the ability to present the ...READ MORE

Good question, glad you brought this up. I ...READ MORE

Hi, it is pretty simple, to be ...READ MORE

Hi. Refer to the below command: import pandas ...READ MORE

You can do it by specifying index. ...READ MORE

There are 4 types of dictionary Empty Integer Mixed Dictionary with ...READ MORE

If it makes for an introductory language ...READ MORE

OR

At least 1 upper-case and 1 lower-case letter

Minimum 8 characters and Maximum 50 characters

Already have an account? Sign in.