摘要： Visible light communication(VLC) employing light-emitting diodes(LED) as light source has become a promising field for underwater wireless communication. However, due to the complex underwater environment and scattering, absorption and diffusion effects during optical transmission in water, the underwater VLC system will suffer from poor SNR and highly nonlinear distortion. In this paper, we proposed and experimentally demonstrated a novel Square Geometrical Shaping(SGS) odd order QAM discrete multitone(DMT) and carrier-less amplitude phase(CAP) modulation based VLC system. Unlike to traditional odd order QAM constellation aiming at the minimum Euclidean distance but without Gray coding at edge constellation points, the new SGS constellation boasts acceptable Euclidean distance and thoroughly Gray coding for every constellation point. Therefore, the system performance can be effectively enhanced by SGS constellation for low SNR and high nonlinearity cases. Both numerical and experimental results verify that SGS 32QAM and 128QAM constellations can outperform traditional QAM constellation by mitigating nonlinearity in a blue LED based underwater and free space VLC system using DMT modulation or CAP modulation. By employing SGS 128QAM, we successfully demonstrate 2.534Gb/s 1.2meter underwater transmission at the driving voltage of 1.1Vpp.