Polysaccharide-based hydrogels are being used in a wide variety of applications ranging from tissue engineering to food products due to their biocompatibility and the ease of gel formation. In real-life applications, hydrogels can undergo large strain deformation, which may result in structural damage leading to failure. Here, we report the nonlinear rheological properties and failure behavior of alginate hydrogels, a class of polysaccharide hydrogels, synthesized via ionic and covalent crosslinking. Gels with ionic crosslinks or ionic alginate hydrogels are prepared by addition of Ca2+ ions in the aqueous solution of sodium alginate, and the covalently crosslinked alginate gels or chemical alginate hydrogels are obtained via amidation reactions. Because of their structural differences, ionic and chemical alginate hydrogels display different scattering profiles captured by using small angle X-ray scattering (SAXS) technique. Both ionic and chemical alginate hydrogels exhibit strain stiffening behavior when subjected to large amplitude oscillatory shear (LAOS) and the strain-stiffening behavior is accompanied by negative normal stress. A custom-built cavitation rheometer has been utilized to probe the local failure behavior of these gels. The cavitation rheometry captures different defect growth or fracture mechanism in ionic versus chemical alginate hydrogels, even if these two types of gels have a similar linear elastic modulus. Based on the critical pressure for gel fracture, we have provided an estimate of the critical energy release rate.