Seagoing vessels have to undergo regular visual inspections in order to detect the typical defective situations affecting metallic structures, such as cracks and corrosion. These inspections are currently performed manually by ship surveyors at a great cost. Assisting them during the inspection process by means of a fleet of robots capable of defect detection would, without doubt, decrease the inspection cost. In this paper, a novel algorithm for visual detection of defects on vessel structures is presented. It is implemented as a generic framework that can be configured to compute the features that perform better for the inspection at hand. Inspired by the idea of conspicuity, contrast in intensity, color and orientation, and the isotropic symmetry, are the features selected to detect the defective situations in the vessel structures. These features are computed at multiple scales so that the algorithm can effectively detect the defective areas in the images despite the distance from which this has been taken. Additionally, three different combination operators are tested in order to merge the information provided by the single features and improve the detection performance. Several experiments are reported for the different configurations of the detection framework. They provide better classification ratios than the state of the art methods and prove its usability with images collected by a micro-aerial robotic platform intended for vessel visual inspection.