Postprocessing and Formatting of Datasets

[ncoop57]

This issue focuses on collecting ideas and formalizing the postprocessing steps and formatting of data instances for datasets in different categories, e.g., forums, articles, books, etc.

Initial draft of postprocessing:

1. Exact duplication removal
2. Near duplication removal
3. Removal of specific html tags

Questions for formatting:

1. How to format forums?
2. How to format general website articles?
3. How to format books?