"The web" is a web of cross-references; which are expressed as "links". A link is a pointer to information somewhere—this site, another site, a person to email, a video to view. People expect to find links—and you can't possibly put all the information your users will be interested in on your own site, and keep it all up to date, anyway.
Links are also important for search-engine ranking. Google, very clearly the premiere search engine at the moment, uses the structure of links as one of its most important tools for figuring out which sites have good information and which don't. They're also fairly good at finding and punishing attempts to game them. Playing fair works very well generally, and is much safer and much easier in the long run.
Most sites have occasional links embedded in the content as appropriate, either to other articles on the same site, or to other sites. In addition, quite a few sites have pages that are just collections of recommended links. It's a service to point your users to the best resources (in your opinion) on topics related to (or included in) the topic of your site.
Many people worry about "sending customers away" via links. Certainly some thought should be given to where to put links, and what links to put there; some thought should be given to all design decisions. On the other hand, when looking at a product page, say, and considering purchase, a link to the manufacturer's own page of information on the product often helps to clinch sales; especially if it opens in a new window, so the old position is not lost. Other kinds of links may be best accumulated on a links page, where people will not encounter them in the middle of doing something else and perhaps get distracted and not come back.