now refiene the code for removing the duplicates using set() rather than list(),and refine the common code for extracting data from files as a function
The Tidelift Subscription provides access to a continuously curated stream of human-researched and maintainer-verified data on open source packages and their licenses, releases, vulnerabilities, and development practices.