Big Data can be defined as a dataset that contains a huge volume of information which can be analyzed to discover unknown and useful patterns. In fact, Deep Learning is a technique that is used mainly to facilitate the Big Data Analysis by extracting complex abstractions of high level. Nevertheless, data heterogeneity represents a key challenge in the context of Big Data Analysis. Usually, data providers use different techniques to represent the same real-world object. Moreover, they have a lack of methods for the automation of the classification of the input images. Therefore, we have to add the semantic aspect in the classification for enhancing the Big Data Analysis process. In order to meet this requirement, we put forward an approach that allows semantizing the classification using Convolutional Neural Network and Semantic Memory. We apply the pooling operation to reduce the size of the input data. After a filter succession, the convolution maps are concatenated into a 1-D vector and then we use Semantic Memory to classify the input data.