BigML uses the Isolation Forest algorithm to detect anomalies. This algorithm uses an ensemble of randomized trees to generate anomaly scores. The basic idea behind is to overfit decision tree models. Then BigML generates an anomaly score based on how many splits are needed to isolate an instance from the rest of the data points.
Click here to learn the process for detecting the anomalous points of your dataset.