ThingsBoard 核心优化:通过聚合表提升物联网遥测数据统计性能中,使用触发器来对计算当日功率因素的平均值,同时从通用的角度计算了合计、平均、最大、最小、计数,以便适应所有指标的汇总需求。

然而这种方式却无法用于用电量,因为用电量需要分尖峰平谷时段统计合计数。这样触发器就不太适用了,原因有二:

  • 尖峰平谷时段的定义是基于每个客户的JSON,因为遥测数据量大,使用触发器方式,其解析与匹配会极大影响性能。
  • 设备上网后是先自动上报参数,注册设备,然后再由人工分配该设备是哪一个公司的。当分配给某给公司之前,或公司设置尖峰平谷时段之前,触发器无法获得这个数据。

因此,决定采用定时作业(计划任务)的方式来进行汇总,这与查看报表并不冲突,因为月报都是截至昨天的。改装如下

  1. 修改表结构
ALTER TABLE ts_kv_daily ADD COLUMN sum_value_a DOUBLE PRECISION DEFAULT 0;
ALTER TABLE ts_kv_daily ADD COLUMN sum_value_b DOUBLE PRECISION DEFAULT 0;
ALTER TABLE ts_kv_daily ADD COLUMN sum_value_c DOUBLE PRECISION DEFAULT 0;
ALTER TABLE ts_kv_daily ADD COLUMN sum_value_d DOUBLE PRECISION DEFAULT 0;
  1. 增加汇总函数b版本,可以解析尖峰平谷。该函数与put_ts_kv_daily_a的区别是增加了尖峰平谷汇总,因而要复杂一些,目前仅用于有功电量的调用。
CREATE OR REPLACE FUNCTION public.put_ts_kv_daily_b(
    p_date date,
    p_key integer
)
RETURNS void
LANGUAGE 'plpgsql'
COST 100
VOLATILE PARALLEL UNSAFE
AS $BODY$
DECLARE
    v_start_ts BIGINT;
    v_end_ts BIGINT;
    v_daily_ts BIGINT;
BEGIN
    -- 计算时间范围
    v_start_ts := (EXTRACT(EPOCH FROM 
        (p_date::text || ' 00:00:00 Asia/Shanghai')::timestamptz
    ) * 1000)::BIGINT;
    
    v_end_ts := v_start_ts + (24 * 60 * 60 * 1000) - 1;
    v_daily_ts := v_start_ts;
    
    WITH 
    -- 方法1:直接调用函数并显式转换参数类型
    customer_parsed_rules AS (
        SELECT 
            ak.entity_id as customer_id,
            per.month,
            per.time_type,
            per.start_time,
            per.end_time
        FROM attribute_kv ak
        INNER JOIN key_dictionary kd ON kd.key_id = ak.attribute_key
        JOIN public.parse_electric_quantity_rule(ak.json_v::jsonb) per ON TRUE
        WHERE kd.key = '尖峰平谷'
    )
    INSERT INTO ts_kv_daily (
        entity_id, key, ts, sum_value, count_value, avg_value, 
        min_value, max_value, sum_value_a, sum_value_b, 
        sum_value_c, sum_value_d
    )
    SELECT 
        a.entity_id,
        p_key,
        v_daily_ts,
        SUM(COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION)),
        COUNT(*),
        AVG(COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION)),
        MIN(COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION)),
        MAX(COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION)),
        SUM(CASE WHEN cpr.time_type = 'times_a' THEN COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION) ELSE 0 END)::NUMERIC(22,2) as sum_value_a,
        SUM(CASE WHEN cpr.time_type = 'times_b' THEN COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION) ELSE 0 END)::NUMERIC(22,2) as sum_value_b,
        SUM(CASE WHEN cpr.time_type = 'times_c' THEN COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION) ELSE 0 END)::NUMERIC(22,2) as sum_value_c,
        SUM(CASE WHEN cpr.time_type = 'times_d' THEN COALESCE(a.dbl_v, a.long_v::DOUBLE PRECISION) ELSE 0 END)::NUMERIC(22,2) as sum_value_d
    FROM ts_kv a
    INNER JOIN device d ON d.id = a.entity_id
    LEFT JOIN customer_parsed_rules cpr 
        ON cpr.customer_id = d.customer_id
        AND cpr.month = EXTRACT(MONTH FROM to_timestamp(a.ts / 1000))::INT
        AND TO_CHAR(to_timestamp(a.ts / 1000), 'HH24:MI:SS')::TIME BETWEEN cpr.start_time AND cpr.end_time
    WHERE a.key = p_key
      AND a.ts >= v_start_ts
      AND a.ts <= v_end_ts
      AND (a.dbl_v IS NOT NULL OR a.long_v IS NOT NULL)
    GROUP BY a.entity_id
    ON CONFLICT (entity_id, key, ts) DO UPDATE SET
        sum_value = EXCLUDED.sum_value,
        count_value = EXCLUDED.count_value,
        avg_value = EXCLUDED.avg_value,
        min_value = EXCLUDED.min_value,
        max_value = EXCLUDED.max_value,
        sum_value_a = EXCLUDED.sum_value_a,
        sum_value_b = EXCLUDED.sum_value_b,
        sum_value_c = EXCLUDED.sum_value_c,
        sum_value_d = EXCLUDED.sum_value_d;
    
    RAISE NOTICE '日统计完成: 日期=%, key=%', p_date, p_key;
END;
$BODY$;
  1. 设置定时作业,使用pgAgent在每天凌晨2点对昨天的功率因素和有功电量进行汇总。设置定时作业的方法参见ThingsBoard - 执行定时任务
-- 执行功率因素聚合函数
SELECT put_ts_kv_daily_a(CURRENT_DATE - 1, 96);
-- 执行有功电量聚合函数
SELECT put_ts_kv_daily_b(CURRENT_DATE - 1, 98);
  1. 现在可以删除以前的触发器了。
-- 先删除触发器,再删除函数
DROP TRIGGER IF EXISTS trg_update_daily_stats ON ts_kv;

-- 然后删除函数
DROP FUNCTION IF EXISTS update_ts_kv_daily();
  1. 可以在第二天查询结果
--在postgres库查询作业日志
select * From pgagent.pga_joblog
select * From pgagent.pga_jobsteplog
--查询结果表
select * From ts_kv_daily where key=98 and to_timestamp(ts / 1000) = '2026-01-05'
Logo

腾讯云面向开发者汇聚海量精品云计算使用和开发经验,营造开放的云计算技术生态圈。

更多推荐