The accumulation of rare-earth and heavy metal ions from wastewater is important for industrial technology. However, practical accumulators of metal ions are expensive with respect procurement of raw materials, synthesis, and preparation. Therefore, it is preferable to accumulate metal ions using sustainable resources, such as natural polymers. Fucoidan, a water-soluble natural polymer, is a sulfated polysaccharide from the cell-wall of brown algae. Therefore, fucoidan behaves as an acidic polysaccharide in an aqueous solution. We prepared a fucoidan-inorganic composite material by mixing fucoidan and a silane coupling reagent, bis(3-(trimethoxysilyl)propyl)amine (SiNSi). This fucoidan-SiNSi (F-SiNSi) composite material showed a water-insoluble property. This is due to the encapsulation of fucoidan into a three-dimensional network of SiNSi with siloxane bonding. When the F-SiNSi composite material is immersed in a metal ion-containing aqueous solution, the composite material accumulated the metal ions. The binding affinity of each metal ion was Ca(II), Mg(II) << Nd(III) < Cu(II), Zn(II), Ni(II), La(III) < In(III) < Y(III). Additionally, the maximum-accumulated amounts of the Nd(III), Cu(II), Zn(II), Ni(II), La(III), In(III), and Y(III) ions were 140, 200, 190, 200, 200, 230, and 270 nmol per mg of fucoidan, respectively. Furthermore, the molar ratios of the acidic groups (the sulfate and carboxyl groups) in the fucoidan and accumulated metal ions, were 0.081–0.156. Therefore, the F-SiNSi composite material showed a selectivity for rare-earth and heavy metal ions. The accumulation mechanism of the rare-earth and heavy metal ions was related to the carboxyl groups in the fucoidan.