The project uses the Fake and Real News Dataset from Kaggle, containing more than 44,000 news articles labeled as either fake or real. The large size of the dataset provides sufficient examples for training and evaluating a text classification model.
The text data was preprocessed by converting all characters to lowercase, removing punctuation, and eliminating extra whitespace. The article titles and main text were combined into a single feature to provide more contextual information to the model.
TF-IDF vectorization was used to transform textual data into numerical features. This method assigns higher importance to words that are distinctive within the dataset while reducing the influence of commonly occurring terms.
A Logistic Regression classifier was selected because it performs effectively on high-dimensional sparse text data, is computationally efficient, and offers good interpretability.
The model was evaluated using accuracy, F1-score, and a classification report containing precision and recall metrics. These measures provide a comprehensive understanding of the classifier's performance on unseen data.
- The dataset labels are assumed to be correct.
- Only English-language news articles are considered.
- Basic preprocessing techniques are sufficient without advanced NLP methods such as stemming or lemmatization.