Some people shall be running a Clickhouse-server in an environment with less than 32GB of system memory for testing purposes or a smaller setup at home.
it can be wishful to tune Clickhouse to this situation so it does not run out of memory or eat up lots of system resources. This setup is based on a installation of Clickhouse on Rocky Linux 8 or 9, with running docker and Coroot ont the same system.
To do this create a file: /etc/clickhouse-server/config.d/z_log_disable.xml
Put the following in the file:
<?xml version="1.0"?>
<clickhouse>
<asynchronous_metric_log remove="1"/>
<metric_log remove="1"/>
<latency_log remove="1"/>
<query_thread_log remove="1" />
<query_log remove="1" />
<query_views_log remove="1" />
<part_log remove="1"/>
<session_log remove="1"/>
<text_log remove="1" />
<trace_log remove="1"/>
<crash_log remove="1"/>
<opentelemetry_span_log remove="1"/>
<zookeeper_log remove="1"/>
</clickhouse>
Smaller logging setup:
When running a default setup, Clickhouse can take up a lot of data for logging to logfiles what one might not need. You can adjust the logging level and the amount of logging data to keep in /etc/clickhouse-server/config.xml
Change the log level from debug to information (or even none):
Change the log level from debug to information (or even none):
<levels>
<ConfigReloader>information</ConfigReloader>
</levels>
Change the amount of logging data (default 1000M) and number of files tot keep (default 10)
<size>100M</size>
<count>7</count>
<ConfigReloader>information</ConfigReloader>
</levels>
Change the amount of logging data (default 1000M) and number of files tot keep (default 10)
<size>100M</size>
<count>7</count>
Smaller database logging setup:
Clickhouse keeps it's logging in the database to and this can be an extensive amount of data (20GB). Logging to the database can be completely disabled.
To do this create a file: /etc/clickhouse-server/config.d/z_log_disable.xml
Put the following in the file:
<?xml version="1.0"?>
<clickhouse>
<asynchronous_metric_log remove="1"/>
<metric_log remove="1"/>
<latency_log remove="1"/>
<query_thread_log remove="1" />
<query_log remove="1" />
<query_views_log remove="1" />
<part_log remove="1"/>
<session_log remove="1"/>
<text_log remove="1" />
<trace_log remove="1"/>
<crash_log remove="1"/>
<opentelemetry_span_log remove="1"/>
<zookeeper_log remove="1"/>
</clickhouse>
Clickhouse-server memory adjustments:
Adjust the mark cache size in /etc/clickhouse-server/config.xml:
<!-- <mark_cache_size>5368709120</mark_cache_size> -->
<mark_cache_size>2684354560</mark_cache_size>
<!-- <mark_cache_size>5368709120</mark_cache_size> -->
<mark_cache_size>2684354560</mark_cache_size>
Adjust memory usage in /etc/clickhouse-server/config.xml:
<max_server_memory_usage_to_ram_ratio>0.9</max_server_memory_usage_to_ram_ratio>
<max_server_memory_usage_to_ram_ratio>1.2</max_server_memory_usage_to_ram_ratio>
<max_server_memory_usage_to_ram_ratio>0.9</max_server_memory_usage_to_ram_ratio>
<max_server_memory_usage_to_ram_ratio>1.2</max_server_memory_usage_to_ram_ratio>
After al these changes restart clickhouse-server and your setup will be running smooth in a smaller environment, in my case this is a 4 core 16GB node.
Reacties
Een reactie posten