ben
/
sudoisbot
1
1
Fork 1
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

fixing slowness when screen_pub queries the db #4

Merged
ben merged 7 commits from fix-db-read into main 2022-09-18 14:17:00 +00:00
Conversation 2 Commits 7 Files Changed 6 +57 -46
ben commented 2022-09-18 12:38:06 +00:00
Owner
Copy Link
No description provided.
ben added 1 commit 2022-09-18 12:38:07 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
8975664d0c select temp by id from db
ben commented 2022-09-18 12:50:07 +00:00
Author
Owner
Copy Link
    raise ImproperlyConfigured('MySQL driver not installed!')
peewee.ImproperlyConfigured: MySQL driver not installed!

using mysql with peewee: http://docs.peewee-orm.com/en/latest/peewee/database.html#using-mysql

  • pymysql: pure python, slower
  • mysqlclient: uses a c-extension, faster. module name: MySQLdb depends on apt-get packages:
    • python3-dev
    • default-libmysqlclient-dev
    • build-essential
```python raise ImproperlyConfigured('MySQL driver not installed!') peewee.ImproperlyConfigured: MySQL driver not installed! ``` using mysql with peewee: http://docs.peewee-orm.com/en/latest/peewee/database.html#using-mysql * `pymysql`: pure python, slower * `mysqlclient`: uses a c-extension, faster. module name: `MySQLdb` depends on `apt-get` packages: * `python3-dev` * `default-libmysqlclient-dev` * `build-essential`
ben added 1 commit 2022-09-18 12:53:02 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
f2904f0a8a install pymysql and upgrade peewee
ben added 3 commits 2022-09-18 13:19:39 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
4fe950859d flake8 fixes
ben added 2 commits 2022-09-18 13:44:34 +00:00
ben/sudoisbot/pipeline/head This commit looks good Details
ben/sudoisbot/pipeline/pr-main This commit looks good Details
882b5e5525 adjust cwd and fix entrypoint in Dockerfile to run correctly
ben commented 2022-09-18 14:06:34 +00:00
Author
Owner
Copy Link

it would take 5-6 seconds (and lots of disk IO) per query, so it would take 25-30s to generate one message with my 5 sensors:

query for: 6.241361141204834s
query for: 5.458697319030762s
query for: 6.028265714645386s
query for: 6.01287579536438s
query for: 5.792287111282349s

now it takes ~0.0007s per query, so it take ~0.002s to generate one message:

query for: 0.0009050369262695312s
query for: 0.0007789134979248047s
query for: 0.0007634162902832031s
query for: 0.0007436275482177734s
query for: 0.0007658004760742188s
it would take 5-6 seconds (and lots of disk IO) per query, so it would take 25-30s to generate one message with my 5 sensors: ``` query for: 6.241361141204834s query for: 5.458697319030762s query for: 6.028265714645386s query for: 6.01287579536438s query for: 5.792287111282349s ``` now it takes ~0.0007s per query, so it take ~0.002s to generate one message: ``` query for: 0.0009050369262695312s query for: 0.0007789134979248047s query for: 0.0007634162902832031s query for: 0.0007436275482177734s query for: 0.0007658004760742188s ```
ben merged commit f962b6d8c8 into main 2022-09-18 14:17:00 +00:00
ben deleted branch fix-db-read 2022-09-18 14:17:00 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: ben/sudoisbot#4
No description provided.
Delete Branch "fix-db-read"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?