fixing slowness when screen_pub queries the db #4

Merged
ben merged 7 commits from fix-db-read into main 2022-09-18 14:17:00 +00:00
Owner
No description provided.
ben added 1 commit 2022-09-18 12:38:07 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
8975664d0c
select temp by id from db
Author
Owner
    raise ImproperlyConfigured('MySQL driver not installed!')
peewee.ImproperlyConfigured: MySQL driver not installed!

using mysql with peewee: http://docs.peewee-orm.com/en/latest/peewee/database.html#using-mysql

  • pymysql: pure python, slower
  • mysqlclient: uses a c-extension, faster. module name: MySQLdb depends on apt-get packages:
    • python3-dev
    • default-libmysqlclient-dev
    • build-essential
```python raise ImproperlyConfigured('MySQL driver not installed!') peewee.ImproperlyConfigured: MySQL driver not installed! ``` using mysql with peewee: http://docs.peewee-orm.com/en/latest/peewee/database.html#using-mysql * `pymysql`: pure python, slower * `mysqlclient`: uses a c-extension, faster. module name: `MySQLdb` depends on `apt-get` packages: * `python3-dev` * `default-libmysqlclient-dev` * `build-essential`
ben added 1 commit 2022-09-18 12:53:02 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
f2904f0a8a
install pymysql and upgrade peewee
ben added 3 commits 2022-09-18 13:19:39 +00:00
ben added 2 commits 2022-09-18 13:44:34 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
ben/sudoisbot/pipeline/pr-main This commit looks good Details
882b5e5525
adjust cwd and fix entrypoint in Dockerfile to run correctly
Author
Owner

it would take 5-6 seconds (and lots of disk IO) per query, so it would take 25-30s to generate one message with my 5 sensors:

query for: 6.241361141204834s
query for: 5.458697319030762s
query for: 6.028265714645386s
query for: 6.01287579536438s
query for: 5.792287111282349s

now it takes ~0.0007s per query, so it take ~0.002s to generate one message:

query for: 0.0009050369262695312s
query for: 0.0007789134979248047s
query for: 0.0007634162902832031s
query for: 0.0007436275482177734s
query for: 0.0007658004760742188s
it would take 5-6 seconds (and lots of disk IO) per query, so it would take 25-30s to generate one message with my 5 sensors: ``` query for: 6.241361141204834s query for: 5.458697319030762s query for: 6.028265714645386s query for: 6.01287579536438s query for: 5.792287111282349s ``` now it takes ~0.0007s per query, so it take ~0.002s to generate one message: ``` query for: 0.0009050369262695312s query for: 0.0007789134979248047s query for: 0.0007634162902832031s query for: 0.0007436275482177734s query for: 0.0007658004760742188s ```
ben merged commit f962b6d8c8 into main 2022-09-18 14:17:00 +00:00
ben deleted branch fix-db-read 2022-09-18 14:17:00 +00:00
Sign in to join this conversation.
No reviewers
No Label
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: ben/sudoisbot#4
No description provided.