T

wjones df89aad7e6 Add MongoDB integration and JSON download functionality

- Created a new script `update_mongodb.py` to handle downloading JSON data from APIs and loading it into MongoDB.
- Implemented functions for downloading JSON files, loading data into MongoDB collections, and managing updates.
- Updated `update_json.py` to instantiate the `dbmongo` class for MongoDB operations.
- Enhanced logging for better tracking of the download and load processes.

2026-02-05 09:02:37 -05:00

api

repo migration

2025-05-20 12:02:07 -04:00

cache

Refactor code structure for improved readability and maintainability

2025-11-14 16:24:06 -05:00

Comment out unused code in CreateTempTables and update_tvupdates methods for future verification; streamline set_apienv function by removing jget parameter and related logic.

2025-12-12 13:48:05 -05:00

logs

repo migration

2025-05-20 12:02:07 -04:00

settings

Add function to update update table

2026-01-22 12:36:47 -05:00

url

repo migration

2025-05-20 12:02:07 -04:00

.gitattributes

repo migration

2025-05-20 12:02:07 -04:00

.gitignore

update gitignore file

2026-02-05 07:31:43 -05:00

json2postgres.py

fix code for all tables in json2postgres.py so all tables get processed now. Update requirements.txt to include pandas.

2025-06-25 16:13:34 -04:00

json_downloader.py

Add MongoDB integration and JSON download functionality

2026-02-05 09:02:37 -05:00

Makefile

Add PostgreSQL virtual table extension and SQLite utilities

2025-12-11 18:45:49 +00:00

mongo2pgsql.py

Refactor update_mongo.py to enhance JSON downloading and file handling; update MongoDB host in test_tableddl_from_yaml.py

2025-12-11 17:53:41 -05:00

README.md

working on json2postgres.py file

2025-06-25 09:17:45 -04:00

requirements.txt

Update psycopg2 dependency to psycopg2-binary in requirements.txt; refine timestamp handling in main function of update_mongo.py; add new cache.tgz file.

2026-01-22 14:25:53 -05:00

run_update_mongo.sh

update gitignore file

2026-02-05 07:31:43 -05:00

update_json.py

Add MongoDB integration and JSON download functionality

2026-02-05 09:02:37 -05:00

update_mongodb.py

Add MongoDB integration and JSON download functionality

2026-02-05 09:02:37 -05:00

README.md

DBSyncer

This project has two parts: 1.) it uses the tvmaze API to download Television dat and store it in a mongodb database. 2.) Takes saved json data and inserts/updates data in a postgresql database.

Learn more about creating GitLab projects.