This study proposes a novel approach to evaluate the integration of solar photovoltaic (PV) and wind turbine renewable energy systems (RES) with Electrolyzer-Fuel Cell Energy Storage System (EFCS) and Battery Energy Storage System (BESS). The objective is to minimize the weighted average cost of energy (waCOE) while maintaining a specific renewable energy fraction (FRES). To achieve this objective, a new hybrid optimization system that combines Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) is proposed to simultaneously optimize the capacity of PV, wind turbine, battery, electrolyzer, hydrogen tank, and fuel cell, which makes it a complex and nonlinear optimization problem. The proposed method produces more practical outcomes by implementing some constraints on PSO and strengthens its ability to exploit optimization opportunities. Six scenarios representing different integrated systems are investigated and compared technically and economically, including PV + BESS, PV + EFCS, Wind + BESS, Wind + EFCS, PV + Wind + BESS, and PV + Wind + EFCS. A case study is conducted on a university campus situated on a Mediterranean island, where electricity production relies on imported fossil fuel. The findings indicate that a PV-wind hybrid system consisting of a 1.15 MW wind turbine, a 2.89 MW PV, and a 2.31 MWh BESS has the lowest waCOE of 0.1838 €/kWh. This system is capable of providing a demand–supply fraction (DSF) of 44.63% with a FRES of 60.1%. The study found that a hybrid system consisting of PV and wind turbines showed promise in the examined region due to their complementary characteristics. Additionally, combining various RES with BESS with approximately 33–35% lower waCOE was found to be more economically viable and resulted in 16–20% higher DSF compared to EFCS.