This paper presents the design of an end-to-end adaptive smoothing and TCP-friendly transmission for stored MPEG-4 fine-grained scalable (FGS) videos over the best-effort Internet. The goal is to minimize the playback quality variation when the network conditions are constantly varying. A novel framework for FGS video delivery over a TCP-friendly connection is first presented. In the context of this scheme, and under the assumption of complete knowledge of bandwidth evolution, an offline quality adaptive smoothing algorithm is derived, and an online adaptive smoothing algorithm is also developed based on the predicted available bandwidth to stream FGS video over the TCP-friendly rate control (TFRC) Protocol with the enhanced ARAR model. Through simulation experiments, it has been shown that the online adaptive algorithm performs almost as well as the offline version for a wide-range of the bandwidth scenarios, and a smooth and TCP-friendly video transfer can be accomplished by the proposed scheme.