华为云云耀云服务器L实例评测｜基于canal缓存自动更新流程 &amp； SpringBoot项目应用案例和源码

2401_84004024

261人浏览 · 2024-05-02 16:16:08

2401_84004024 · 2024-05-02 16:16:08 发布

如果数据库的数据更新，canal监听到，发现是缓存对应的表，并且是对应的字段发生变化，则进行缓存的自动更新

在这里插入图片描述

缓存自动同步更新

在这里插入图片描述

存到Redis里面的最新的数据

在这里插入图片描述

4.缓存更新前端展示

缓存更新后，前端再次查询，获得最新的数据

在这里插入图片描述

核心代码源码

1.配置yml和配置类

配置yml文件

server:
  port: 10050

## 是否启用安全框架 true为开启，false为关闭
security:
  isOpen: true

## 是否开启canal管道，true为开启，false为关闭
canal:
  isOpen: true

# canal的相关配置
canalConfig:
  host: 124.70.138.34
  port: 11111

在这里插入图片描述

Redis存Java对象的配置类

package com.tianju.fresh.config.redis;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;

@Configuration
public class RedisSerializeConfig {
    @Bean
    public RedisTemplate redisTemplateInit(RedisConnectionFactory redisConnectionFactory) {

        RedisTemplate<String,Object> redisTemplate = new RedisTemplate<>();

        redisTemplate.setConnectionFactory(redisConnectionFactory);
        //设置序列化Key的实例化对象
        redisTemplate.setKeySerializer(new StringRedisSerializer());
        //设置序列化Value的实例化对象
        redisTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());

        /\*\*
 \*
 \* 设置Hash类型存储时，对象序列化报错解决
 \*/
        redisTemplate.setHashKeySerializer(new StringRedisSerializer());
        redisTemplate.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
        return redisTemplate;
    }
}

2.canal自动更新代码

canal自动更新的代码，用canal管道监听MySQL数据变化，自动更新redis缓存

package com.tianju.fresh.config.redis;


import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;


import com.alibaba.otter.canal.client.CanalConnectors;
import com.alibaba.otter.canal.client.CanalConnector;
import com.alibaba.otter.canal.common.utils.AddressUtils;
import com.alibaba.otter.canal.protocol.Message;
import com.alibaba.otter.canal.protocol.CanalEntry.Column;
import com.alibaba.otter.canal.protocol.CanalEntry.Entry;
import com.alibaba.otter.canal.protocol.CanalEntry.EntryType;
import com.alibaba.otter.canal.protocol.CanalEntry.EventType;
import com.alibaba.otter.canal.protocol.CanalEntry.RowChange;
import com.alibaba.otter.canal.protocol.CanalEntry.RowData;
import com.baomidou.mybatisplus.annotation.TableField;
import com.tianju.fresh.entity.common.GoodsTypeVo;
import com.tianju.fresh.entity.common.WarehouseVo;
import com.tianju.fresh.entity.goods.GoodsType;
import com.tianju.fresh.mapper.goods.GoodsTypeMapper;
import com.tianju.fresh.mapper.warehouse.StorehouseMapper;
import com.tianju.fresh.service.common.CommonStaticMethod;
import com.tianju.fresh.util.Constance;
import com.tianju.fresh.util.RedisUtil;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.stereotype.Component;

/\*\*
 \* 用canal管道监听MySQL数据变化，自动更新redis缓存
 \*/
@Slf4j
@Component
public class AutoUpdateRedis {

    @Value("${canalConfig.host}")
    private String host;

    @Value("${canalConfig.port}")
    private Integer port;

    @Autowired
    private RedisTemplate<String,Object> redisTemplate;

    @Autowired
    private GoodsTypeMapper typeMapper;

    @Autowired
    private StorehouseMapper storehouseMapper;

    @Autowired
    private RedisUtil redisUtil;


    public void run() {
        // 创建链接
        final InetSocketAddress HOST = new InetSocketAddress(host,port);
// final InetSocketAddress HOST = new InetSocketAddress("192.168.111.130",11111);
        CanalConnector connector = CanalConnectors.newSingleConnector(HOST, "example", "", "");
        int batchSize = 1000;
        int emptyCount = 0;
        try {
            connector.connect();
            connector.subscribe(".\*\\..\*");
            connector.rollback();
            int totalEmptyCount = 120;
// while (emptyCount < totalEmptyCount) {
            while (true) {
                Message message = connector.getWithoutAck(batchSize); // 获取指定数量的数据
                long batchId = message.getId();
                int size = message.getEntries().size();
                if (batchId == -1 || size == 0) {
                    emptyCount++;
                    if(emptyCount % 100==0 || emptyCount==1){
                        System.out.println("empty count : " + emptyCount);
                    }
                    try {
                        Thread.sleep(1000);
                    } catch (InterruptedException e) {
                    }
                } else {
                    emptyCount = 0;
                    // System.out.printf("message[batchId=%s,size=%s] \n", batchId, size);
                    printEntry(message.getEntries());
                }

                connector.ack(batchId); // 提交确认
                // connector.rollback(batchId); // 处理失败, 回滚数据
            }

// System.out.println("empty too many times, exit");
        } finally {
            connector.disconnect();
        }
    }

    private void printEntry(List<Entry> entrys) {
        for (Entry entry : entrys) {
            if (entry.getEntryType() == EntryType.TRANSACTIONBEGIN || entry.getEntryType() == EntryType.TRANSACTIONEND) {
                continue;
            }

            RowChange rowChage = null;
            try {
                rowChage = RowChange.parseFrom(entry.getStoreValue());
            } catch (Exception e) {
                throw new RuntimeException("ERROR ## parser of eromanga-event has an error , data:" + entry.toString(),
                        e);
            }

            EventType eventType = rowChage.getEventType();
            System.out.println(String.format("================&gt; binlog[%s:%s] , name[%s,%s] , eventType : %s",
                    entry.getHeader().getLogfileName(), entry.getHeader().getLogfileOffset(),
                    entry.getHeader().getSchemaName(), entry.getHeader().getTableName(),
                    eventType));

            String tableName = entry.getHeader().getTableName();
            if (Constance.LISTEN\_TAB\_NAMES.contains(tableName)){
                for (RowData rowData : rowChage.getRowDatasList()) {
                    if (eventType == EventType.DELETE){
                        // 删除之前
                        log.debug("-------删除之前before");
                        changeBefore(rowData.getBeforeColumnsList());

                        log.debug("-------删除之后after");
                        // 传修改的表名，和常量中的map对比，从而对应的redis里的key
// changeAfter(rowData.getAfterColumnsList(),tableName);

                    }else if (eventType == EventType.INSERT){
                        // 插入之前
                        log.debug("-------插入之前before");
                        changeBefore(rowData.getBeforeColumnsList());

                        log.debug("-------插入之后after");
                        // 传修改的表名，和常量中的map对比，从而对应的redis里的key
// changeAfter(rowData.getAfterColumnsList(),tableName);

                    }else {
                        // 修改之前
                        log.debug("-------修改之前before");
                        changeBefore(rowData.getBeforeColumnsList());

                        log.debug("-------修改之后after");
                        // 传修改的表名，和常量中的map对比，从而对应的redis里的key
                        changeAfter(rowData.getAfterColumnsList(),tableName);
                    }
                }
            }
        }
    }


    /\*\*
 \* 数据库更新之前
 \* @param columns
 \*/

    private  void changeBefore(List<Column> columns) {
        for (Column column : columns) {
            System.out.println(column.getName() + " : " + column.getValue() + " update=" + column.getUpdated());
        }
    }

    private  void changeAfter(List<Column> columns, String tableName) {
        for (Column column : columns) {
            System.out.println(column.getName() + " : " + column.getValue() + " update=" + column.getUpdated());

            // 如果是商品类别表变化

            if ("name".equals(column.getName()) && Constance.GOODS\_TYPE\_TAB\_NAME.equals(tableName)){ // 默认是名称那一列变化才更新缓存
                // 先删除key，再设置好新的key
                Map tabNameToRedisKey = Constance.getTabNameToRedisKey();
                String redisKey = (String) tabNameToRedisKey.get(tableName);
                redisTemplate.delete(redisKey);
                // TODO:设置新的key
                List<GoodsTypeVo> list = CommonStaticMethod.getGoodsTypeVos(typeMapper);
                redisUtil.saveObjectToRedis(Constance.GOODS\_TYPE\_REDIS\_KEY, list);
                break;
            }

            // 如果是仓库那张表变化 storehouse
            if ("name".equals(column.getName()) && Constance.STOREHOUSE\_TAB\_NAME.equals(tableName)){ // 默认是名称那一列变化才更新缓存
                // 先删除key，再设置好新的key
                Map tabNameToRedisKey = Constance.getTabNameToRedisKey();
                String redisKey = (String) tabNameToRedisKey.get(tableName);
                redisTemplate.delete(redisKey);

                // 设置新的key，存到redis里面
                List<WarehouseVo> list = CommonStaticMethod.getStorehouseVos(storehouseMapper);

                redisUtil.saveObjectToRedis(Constance.STOREHOUSE\_REDIS\_KEY,list);

                break;
            }
        }
    }
}

在这里插入图片描述

redis的工具类

package com.tianju.fresh.util;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Component;

@Component
public class RedisUtil {

    @Autowired
    private RedisTemplate<String,Object> redisTemplate;

    public void  saveObjectToRedis(String key,Object json){
        redisTemplate.opsForValue().set(key,json);
    }

    /\*\*
 \* 从redis里面获取json对象，如果没有，返回null
 \* @param key
 \* @return
 \*/
    public Object getJsonFromRedis(String key){
        return redisTemplate.opsForValue().get(key);
    }

}

监听数据库表，列名的常量类

package com.tianju.fresh.util;

import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/\*\*
 \* 专门放各种常量
 \*/
public interface Constance {
    // 设置哪些数据库表需要监听，比如单位unit，仓库warehouse，商品类型等
    List<String> LISTEN\_TAB\_NAMES = Arrays.asList("goods\_type","unit\_commodity","warehouse\_center","warehouse\_tab");

    String GOODS\_TYPE\_TAB\_NAME = "goods\_type";
    String GOODS\_TYPE\_REDIS\_KEY = "goodsTypeVo";
    String UNIT\_TAB\_NAME = "unit\_commodity";
    String UNIT\_REDIS\_KEY = "unitVo";
    String WAREHOUSE\_TAB\_NAME = "warehouse\_center";
    String WAREHOUSE\_REDIS\_KEY = "warehouseCenterVo";
    String STOREHOUSE\_TAB\_NAME = "warehouse\_tab";
    String STOREHOUSE\_REDIS\_KEY = "storehouseVo";

    static Map getTabNameToRedisKey(){
        Map<String,String> map = new HashMap<>();
        map.put(GOODS\_TYPE\_TAB\_NAME,"goodsTypeVo");
        map.put("unit\_commodity","unitVo");
        map.put("warehouse\_center","warehouseCenterVo");
        map.put("warehouse\_tab","storehouseVo");
        return map;
    }
}

3.查询的业务层service代码

在这里插入图片描述

    @Override
    public List<WarehouseVo> findStorehouse() {
// List<Storehouse> storehouses = storehouseMapper.selectList(null);
// List<WarehouseVo> list = new ArrayList<>(storehouses.size());
// storehouses.forEach(s->{
// list.add(new WarehouseVo(s.getId()+"", s.getName()));
// });

        // 是否有小仓库的redis的key
        Boolean hasKey = redisTemplate.hasKey(Constance.STOREHOUSE\_REDIS\_KEY);
        if (hasKey){ // 如果有，走缓存
            List<WarehouseVo> list = (List<WarehouseVo>) redisTemplate.opsForValue()
                    .get(Constance.STOREHOUSE\_REDIS\_KEY);
            log.debug("get storehouseVo from redis: "+list);
            return list;
        }
        // 如果没有从数据库查询，存到redis里面
        List<WarehouseVo> list = CommonStaticMethod.getStorehouseVos(storehouseMapper);
        log.debug("get storehouseVo from mysql: "+list);
        redisUtil.saveObjectToRedis(Constance.STOREHOUSE\_REDIS\_KEY, list);
        return list;
    }

4.主启动类

主启动类实现implements CommandLineRunner方法，启动canal通道，进行监听数据库的变化，实现缓存同步更新

package com.woniu.fresh;


import com.woniu.fresh.config.redis.AutoUpdateRedis;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.CommandLineRunner;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.scheduling.annotation.EnableScheduling;


@SpringBootApplication
@Slf4j
@EnableAspectJAutoProxy // 让动态代理生效
@EnableScheduling // 让定时任务生效
public class FreshApp implements CommandLineRunner {
    public static void main(String[] args) {
        SpringApplication.run(FreshApp.class);
    }

    @Autowired
    private AutoUpdateRedis autoUpdateRedis;

    @Value("${canal.isOpen}")
    private Boolean isCanal;

    @Override
    public void run(String... args) throws Exception {

        if (isCanal){
            log.debug(">>>>>启动缓存自动更新");
            autoUpdateRedis.run();
        }
    }
}

5.前端vue代码

<template>
    <div>
        <el-row>
            <el-col :span="24">
                <el-form :inline="true" label-width="80px">
                    <el-form-item label="领料单号">
                        <el-input v-model="findGoodsParams.shoppingNo"></el-input>
                    </el-form-item>

                    <el-form-item label="领料数量≥">
                        <el-input v-model="findGoodsParams.nums"></el-input>
                    </el-form-item>

                    <el-form-item label="原料名称">
                        <el-input v-model="findGoodsParams.rawName"></el-input>
                    </el-form-item>

                    <el-form-item label="领料仓库">
                        <el-select v-model="findGoodsParams.warehouseId" placeholder="请选择">
                            <el-option v-for="item in commondata.storehouse" :key="item.value" :label="item.label"
                                :value="item.value">
                            </el-option>
                        </el-select>
                    </el-form-item>

                    <el-form-item label="状态">
                        <el-select v-model="findGoodsParams.status" placeholder="请选择">
                            <el-option v-for="item in commondata.status" :key="item.value" :label="item.label"
                                :value="item.value">
                            </el-option>
                        </el-select>
                    </el-form-item>

                    <el-form-item>
                        <el-button type="primary" @click="findGoods">查询</el-button>
                        <el-button type="success" @click="reFindGoods">重置</el-button>
                    </el-form-item>
                </el-form>
            </el-col>
        </el-row>

        <el-row>
            <el-col :span="24">
                <el-button type="success" @click="addGoodsBtn">新增</el-button>
                <el-button type="warning" icon="el-icon-edit" @click="batchDelete">批量审批</el-button>
                <el-button type="primary" icon="el-icon-magic-stick" @click="batchPickDoing">批量领取中</el-button>
                <el-button type="primary" icon="el-icon-shopping-cart-full" @click="batchPickDown">批量已领完</el-button>

            </el-col>
        </el-row>

        <el-row>
            <el-col :span="24">
                <el-table :data="tableData" style="width: 100%" @selection-change="handleSelectionChange">
                    <el-table-column type="selection" width="55">
                    </el-table-column>
                    <el-table-column prop="pickNo" label="领料单号" width="240">
                    </el-table-column>
                    <el-table-column prop="name" label="原材料名" width="100">
                    </el-table-column>
                    <el-table-column prop="nums" label="领料数量" width="80">
                    </el-table-column>
                    <el-table-column prop="unit" label="单位" width="80">
                    </el-table-column>
                    <el-table-column prop="warehouse" label="领料仓库" width="180">
                    </el-table-column>
                    <el-table-column prop="emp" label="仓管员" width="150">
                    </el-table-column>

                    <el-table-column prop="status" label="状态" width="80">

                        <template slot-scope="scope">
                            <span v-if="scope.row.status == '0'" style="color: red;">未审批</span>
                            <span v-if="scope.row.status == '1'" style="color: rgb(9, 209, 109);">已审批</span>
                            <span v-if="scope.row.status == '2'" style="color: rgb(44, 39, 205);">已领取</span>
                            <span v-if="scope.row.status == '3'" style="color: rgb(173, 16, 157);">已领完</span>
                            <!-- {{ scope.row.status == 1 ? '已上架' : '下架' }} -->
                        </template>
                    </el-table-column>
                    <el-table-column label="操作">
                        <template slot-scope="scope">
                            <el-button type="primary" icon="el-icon-edit" circle
                                @click="loadBtn(scope.row.id)">审批</el-button>
                        </template>
                    </el-table-column>
                </el-table>

            </el-col>
        </el-row>
        <el-row>
            <el-col :span="24">
                <div class="block">
                    <el-pagination @size-change="handleSizeChange" @current-change="handleCurrentChange"
                        :current-page.sync="currentPage" :page-sizes="[3, 5, 10]" :page-size="3"
                        layout="total,sizes, prev, pager, next" :total=total>
                    </el-pagination>
                </div>
            </el-col>
        </el-row>


        <!-- 新增原材料入库弹窗 \*\*\*\*\*\*\* -->
        <el-dialog title="添加原材料单" :visible.sync="b">
            <el-form>
                <el-row>
                    <el-col :span="12">

                        <el-form-item label="原料名称" clearable>
                            <el-select v-model="shoppingNoId" placeholder="请选择"
                                style="width: 355px;margin-right:px;margin-left:px" @change="selectBuyNo">
                                <el-option v-for="item in shoppingIdToNo" :key="item.label" :label="item.label"
                                    :value="item.label">
                                </el-option>
                            </el-select>
                        </el-form-item>

                        <el-form-item label="数量" :label-width="formLabelWidth">
                            <el-input v-model="goods.nums" autocomplete="off"></el-input>
                        </el-form-item>



                        <el-form-item label="单位" clearable>
                            <el-select v-model="goods.unit" placeholder="请选择商品单位"
                                style="width: 355px;margin-right:px;margin-left:px">
                                <el-option v-for="item in commondata.unit" :key="item.value" :label="item.label"
                                    :value="item.value">
                                </el-option>
                            </el-select>
                        </el-form-item>

                        <el-form-item label="中心仓库" clearable>
                            <el-select v-model="goods.warehouse" placeholder="请选择" disabled
                                style="width: 355px;margin-right:px;margin-left:px">
                                <el-option v-for="item in commondata.warehouse" :key="item.value" :label="item.label"
                                    :value="item.value">
                                </el-option>
                            </el-select>
                        </el-form-item>

                        <el-form-item label="领料仓库" clearable>
                            <el-select v-model="goods.storehouse" placeholder="请选择"
                                style="width: 355px;margin-right:px;margin-left:px">
                                <el-option v-for="item in commondata.storehouse" :key="item.value" :label="item.label"
                                    :value="item.value">
                                </el-option>
                            </el-select>
                        </el-form-item>


                    </el-col>
                </el-row>

            </el-form>

            <div slot="footer" class="dialog-footer">
                <el-button @click="b = false">取 消</el-button>
                <el-button type="primary" @click="addGoods()">确 定</el-button>
            </div>
        </el-dialog>



    </div>
</template>

<script>
export default {
    data() {
        return {
            findGoodsParams: {
                "pickNo": ""
                , "name": ""
                , "nums": ""
                , "storehouseId": ""
                , "status": ""
            },
            // 批量删除的id
            deleteIds: [],


            tableData: [
                {


### 最后

这份《“java高分面试指南”-25分类227页1000+题50w+字解析》同样可分享给有需要的朋友，感兴趣的伙伴们可挑战一下自我，在不看答案解析的情况，测试测试自己的解题水平，这样也能达到事半功倍的效果！（好东西要大家一起看才香）

![image](https://img-blog.csdnimg.cn/img_convert/5602e048ceef8885db8e91f761fa0b9d.webp?x-oss-process=image/format,png)

![image](https://img-blog.csdnimg.cn/img_convert/943a3b71d602bd22bd9d23cdf1afcfa3.webp?x-oss-process=image/format,png)



tton type="primary" @click="addGoods()">确 定</el-button>
            </div>
        </el-dialog>



    </div>
</template>

<script>
export default {
    data() {
        return {
            findGoodsParams: {
                "pickNo": ""
                , "name": ""
                , "nums": ""
                , "storehouseId": ""
                , "status": ""
            },
            // 批量删除的id
            deleteIds: [],


            tableData: [
                {


### 最后

这份《“java高分面试指南”-25分类227页1000+题50w+字解析》同样可分享给有需要的朋友，感兴趣的伙伴们可挑战一下自我，在不看答案解析的情况，测试测试自己的解题水平，这样也能达到事半功倍的效果！（好东西要大家一起看才香）

[外链图片转存中...(img-o5WmAGjF-1714637757139)]

[外链图片转存中...(img-nuhGFPLB-1714637757140)]



> **本文已被[CODING开源项目：【一线大厂Java面试题解析+核心总结学习笔记+最新讲解视频+实战项目源码】](https://bbs.csdn.net/topics/618154847)收录**

腾讯云开发者社区

腾讯云面向开发者汇聚海量精品云计算使用和开发经验，营造开放的云计算技术生态圈。

更多推荐

Elasticsearch复杂数据类型终极指南：从入门到精通

Elasticsearch作为功能强大的搜索引擎，支持多种复杂数据类型，让开发者能够灵活处理各种结构化和非结构化数据。本文将带你全面了解Elasticsearch中的复杂数据类型，从基础概念到实际应用，助你轻松掌握数据建模的核心技巧。## 内部对象：构建层级化数据结构在Elasticsearch中，对象类型（Object）是最基础的复杂数据类型之一，用于表示具有嵌套关系的数据。例如，我们可

腾讯云开发者社区

终极指南：Flink SQL连接器版本管理从混乱到有序的升级之路

Apache Flink作为流处理领域的佼佼者，其SQL连接器的版本管理一直是开发者面临的核心挑战。本文将系统讲解Flink SQL连接器版本管理的最佳实践，帮助你轻松应对版本兼容性问题，实现从混乱到有序的升级之旅。## 连接器版本管理的常见痛点 😫在Flink应用开发中，连接器版本管理常常让开发者头疼不已。不同版本的连接器可能导致各种兼容性问题，例如API变更、功能差异甚至运行时错误。

腾讯云开发者社区

如何快速搭建Neon无服务器PostgreSQL：面向初学者的完整指南

Neon是一款革命性的无服务器PostgreSQL解决方案，它通过分离存储和计算层，实现了自动扩缩容、类代码式数据库分支以及零级扩展能力。本指南将帮助你从零开始搭建Neon开发环境，体验这款创新数据库的强大功能。## 准备工作：环境要求与依赖项在开始搭建Neon环境前，请确保你的系统满足以下要求：- Linux操作系统（推荐Ubuntu 20.04+或Debian 11+）- Git