Artificial Intelligence (AI): Natural Language Processing

NLP Tools:

HuggingFace	It provides models for a variety of tasks. Some of them are: In NLP: Summarisation, question answering, table question answering, text classification, fill mask, etc. Hugging Face community Hugging Face UF page Hugging Face UFNLP Page (Healthcare NLP)
Natural language Tool Kit (NLTK)	A leading platform for building Python programs to work with natural language data. Over 50 corpora and lexical resources such as WordNet, along with a suite of text processing libraries for classification, tokenization, stemming, tagging, parsing, and semantic reasoning, wrappers for industrial-strength NLP libraries. Tutorials
PyTorch	A Python library for building deep learning (one kind of machine learning) projects. PyTorch is installed on HiperGator.
TensorFlow	A collection of workflows to develop and train models using Python or JavaScript, etc. The tf. data API is for building complex input pipelines from simple, reusable pieces.
Keras	It offers simpler APIs that minimize the number of user actions required for common use cases. Tutorials
Orange	Friendly user interface Official Video Tutorial (Tasks: Word embeddings, sentiment analysis, text classification, text preprocessing, clustering)
CLAMP (Clinical Language Annotation, Modeling, and Processing Toolkit)	Clinical Language Annotation, Modeling, and Processing Toolkit, is a comprehensive clinical Natural Language Processing (NLP) software that enables recognition and automatic encoding of clinical information in narrative patient reports.
ConvoKit (Cornell Conversational Analysis Toolkit)	It contains tools to extract conversational features and analyze social phenomena in conversations.  Demo Link

Corpora:

Did you know UF HiPerGator stores NLP tools/datasets which you can use right away? Learn more: https://help.rc.ufl.edu/doc/NLP

The Pile Corpus from eleuther.ai (/data/ai/text/data/thepile) for language modeling (paper)

AI-news collection ( /data/ai/text/data/news) from Bing, Duckduckgo, and Google search engines. (this set is updated weekly, contact Eric at UFRC ericeric@ufl.edu for more information)

UF Baldwin Library digital collections (/data/ai/text/data/baldwin_library)

Fact-checking datasets: open-sourced FEVER (Fact Extraction and VERification) contains claims labeled as "Supported", "Refuted" or "NotEnoughInfo". Three datasets available (June 2023 checked) from 2018, 2019, and 2021.

Common sense Q and A: NLP-Progress repository to track the progress in Natural Language Processing (NLP), including the datasets and the current state-of-the-art for the most common NLP tasks. 

Kaggle NLP Datasets: Explore, analyze, and share quality data. 

The open parallel corpus (OPUS)

• A growing collection of translated texts from the web. 
• OPUS is based on open source products and the corpus is also delivered as an open content package.
• All pre-processing is done automatically. No manual corrections have been carried out.

Penn Treebank (PTB)

• The project selected 2,499 stories from a three-year Wall Street Journal (WSJ) collection of 98,732 stories for syntactic annotation. 
• suitable for tasks such as POS tagging and syntactic annotations.

CoNLL2003 

• It concentrates on four types of named entities: persons, locations, organizations, and names of miscellaneous entities that do not belong to the previous three groups.

The Stanford Sentiment Treebank(SST) 

• 9645 sentences, sentiments rated on a scale between 1 and 25, where 1 is the most negative and 25 is the most positive.