We experimentally investigate spin-orbit torque and spin pumping in Y3Fe5O12 (YIG)/Pt bilayers with ultrathin insertion layers at the interface. An insertion layer of Cu suppresses both spin-orbit torque and spin pumping, whereas an insertion layer of Ni80Fe20 (permalloy, Py) enhances them in a quantitatively consistent manner with the reciprocity of the two spin transmission processes. However, we observe a large enhancement of Gilbert damping with the insertion of Py that cannot be accounted for solely by spin pumping, suggesting significant spin-memory loss due to the interfacial magnetic layer. Our findings indicate that the magnetization at the YIG-metal interface strongly influences the transmission and depolarization of pure spin current.