Mesh segmentation and annotation using semantics has received an increased interest with the recent democratisation of 3D reconstruction methods. The common approach is to perform this task in two steps, by first segmenting the mesh and then annotating it. However, this approach does not allow one part to take advantage of the other. In image processing, some methods are combining segmentation and annotation, but they are not generic and require implementation adjustments or rewritings for each modification of the expert knowledge. In this work, we describe an original framework that mixes segmentation and annotation while minimizing the required geometric analysis and we give preliminary results showing its feasability. Our framework provides a generic ontology describing object feature concepts (geometry, topology, etc.) and algorithms allowing to detect these concepts. This ontology can be enlarged by any expert to formally describe a specific domain. The formalized domain description is then used to automatically perform the joint segmentation and annotation of objects and their features, by selecting at each step the most relevant algorithm given the previously detected semantics. This methodology has several advantages. Firsly it allows to segment and annotate objects without any knowledge in mesh or image processing by simply describing the object features in terms of ontological concepts. Secondly this framework can be easily reused and applied to different contexts by simply building on our generic ontology. Finally performing the joint segmentation and annotation allows to use in an efficient way the expert knowledge, reducing possible segmentation errors and the computation time by always launching the most efficient algorithm.