Graph anomaly detection have proved very useful in a wide range of domains. For instance, for detecting anomalous accounts on online platforms, intrusions and failures on communication networks or suspicious and fraudulent behaviors on social networks. However, most existing methods often rely on pre-selected features built from the graph, do not necessarily use local information. To overcome these limits, we present CoBaGAD, a Context-Based Graph Anomaly Detector which exploits local information to detect anomalous nodes of a graph in a semi-supervised way. We use Graph Attention Networks (GAT) with our custom attention mechanism to build local features, aggregate them and classify unlabeled nodes into normal or anomaly. Finally, we show that our algorithm is able to detect anomalies with high precision and recall and, outperforms state-of-the-art baselines.