I am not happy with it yet but that is because I want it to be perfect and it never will be but I do find that I engage with content at a larger scale and more varied than I do when I go to a single source. I am using the nltk features from newspaper for key word extraction + the trending sources to monitor a few hundred sources. Currently I store all the meta data + links ( urls ) + wikipedia links in a pandas dataframe ( which is becoming a problem ) and visualize trends and data about news in a jupyter notebook. For the enhanced summaries + named entity extraction I am using spacy (https://spacy.io/) from there I use SPARQL ( https://en.wikipedia.org/wiki/SPARQL ) to query dbpedia (https://en.wikipedia.org/wiki/DBpedia) to augment entity knowledge ( ex: adding data about the size , industry of a company or summary explanations of scientific concepts, etc ). The named entity matching and augmentation is the portion that needs the most work. Newspaper has some nice caching features so I query all sources everyday but only pull in new articles.

I might play around with moving portions of the data into a graph db and some better ways to query based on concepts. Right now I just write python code to query the pandas DB based on different parameters.

Are you happy with your solution ? Can you share a bit more about your pipeline?

And if you don't want your searches to be tracked by Google, Whoogle is an alternative front-end to access Google Search.

https : / / github (dot) com / kroy94 / whoogle-search

Description: Get Google search results, but without any ads, javascript, AMP links, cookies, or IP address tracking. Easily deployable in one click as a Docker app, and customizable with a single config file. Quick and simple to implement as a primary search engine replacement on both desktop and mobile.

You may notice some features are lacking in Whoogle that you would find in Google Search, but for some people it's probably good enough. Whoogle Search includes Google News(as a tab) on the bottom of the search bar after you type in a query and press enter.

Searx is a privacy respecting metasearch engine. You can also search for news there.

Github: https : / / github (dot) com / searx / searx

Searx Instances: https : / / searx (dot) space

Searx Instances(onion link): http : / / searxspbitokayvkhzhsnljde7rqmn7rvoga6e4waeub3h7ug3nghoad(dot)onion/

Disclaimer: It is probably not a good idea to self-host Whoogle/Searx, since it is obvious the traffic to Google/some other non-privacy respecting search engine operator is from one person. But depending on your threat model maybe it can work.

Of course Brave is bad. But if Google isn't crypto-fascist, i don't know what is. They are an integral part of the military industrial complex and are very cozy with governments worldwide. They are also working on the "technopolice" and promoting AI with all of its shortcomings and biases. About search results, specifically, they are based on a secret ranking sauce privileging bad-quality content... i don't know about english language result, but whatever political topic you lookup for, neo-fascist conspiracy websites (égalité et réconciliation, fdesouche...) are always in the top5 results. How could you defend them?

DuckDuckGo is just a meta-engine and stores little information. Whatever appears in Bing (and maybe others) is what will appear in DuckDuckGo.

A privacy forum where you get told to use google because it does a better job with censoring? Good job guys... I'm out... Have fun with your quality news...

i'm a news junkie with a journalism degree and there's really not many options, even google had law trouble with their news, i doubt an indie crawler would do well