mongo-kafka-connect实战案例

​Kafka Connect可以从数据库或应用程序服务 器收集数据到Kafka topic，使数据可用于低延迟的流处理。导出作业可以将数据从Kafka topic传输到二次存储和 查询系统，或者传递到批处理系统以进行离线分析。

什么是mongodb Change Streams?

Change Stream 指数据的变化事件流，MongoDB 从 3.6 版本开始提供订阅数据变更的功能。

mongodb十大常用命令

mongoexport导出数据,mongodb java获取所有数据库

mongo2.6.9升级到mongo3.0的坑

xecutor error: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

mongodb的读写分离使用Replica Sets来实现

om.mongodb.MongoException: not talking to master and retries used up第一种方法：在java代码中调用dbFactory.getDb().slaveOk(); 第二种方法：在java代码中调用 dbFactory.getDb().setReadPreference(ReadPreference.secondaryPreferred());//在复制集中优先读secondary，如果secondary访问不了的时候就从master中读

Convert转变a Secondary to an Arbiter

如果你的副本集一个次副本节点不需要保持数据了，但是为了选举主副本，需要把次副本转变成arbiter 1、修改程序以致以不能从这个次副本查询数据。比如改变读取策略 2、停止次副本 3、在主副本中执行rs.remove("<:port>") 4 mv mongodb mongodb-old 5 mkdir mongodb 6 mongod --port 27021 --dbpath /data/db --replSet rs 7 rs.addArb("host2:27017") 8 验证rs.conf()

mongo副本集常用命令

增加副本集成员 rs.add(

mongostat-Failed: error connecting to db server: no reachable servers

Failed: error connecting to db server: no reachable serversmongodb没有部署副本集的时候执行mongostat查看mongodb状态的时候没有问题。最近将mongodb部署成了副本集，再次执行mongostat的时候，就报错了。解决办法指定要查看mongodb的ip

Deploy a Replica Set部署副本集

Three member replica sets provide enough redundancy冗余 to survive 生存most network partitions and other system failures. These sets also have sufficient capacity 充足能力 for many distributed read operations. Replica sets should always have an odd number of members副本集应该总是有一个奇数的成员. This ensures that elections will proceed smoothly 顺利. For more about designing replica sets, see the Replication overview.

mongodb Create a User Administrator

use admindb.createUser( { user: "siteUserAdmin", pwd: "password", roles: [ { role: "userAdminAnyDatabase", db: "admin" } ] })

Deploy a Replica Set for Testing and Development单机器测试副本集

在一台电脑测试副本集，副本集成员开启不同的端口号27017，设置不同的mongodb数据路径

Read Preference

By default, an application directs its read operations to the primary member in a replica set. Because write operations are issued to the single primary, reading from the primary returns the latest version of a document[1]. For an application that does not require fully up-to-date data, you can improve read throughput or reduce latency by distributing some or all reads to secondary members of the replica set.

Write Concern for Replica Sets

For a replica set, the default write concern confirms write operations only on the primary. You can, however, override this default write concern, such as to confirm write operations on a specified number of the replica set members.

Replica Set Read and Write Semantics

Users may configure read preference on a per-connection basis to prefer that the read operations return results from the secondary members. If clients configure the read preference to permit 许可证secondary reads, read operations can return data from secondary members that have not replicated more recent write operations 没有复制的最近的写操作

Rollbacks During Replica Set Failover

Rollback Limitations 大于300M数据，您必须手动介入以恢复数据

Factors and Conditions that Affect Elections影响选举的因素和条件

Heartbeats心跳 10s Priority Comparisons优先级比较 Network Partitions网络分区

Three Member Replica Sets

​The minimum architecture of a replica set has three members. A three member replica set can have either three members that hold data, or two members that hold data and an arbiter.

Replica Set Deployment Architectures

The architecture of a replica set affects the set’s capacity and capability. This document provides strategies for replica set deployments and describes common architectures. The standard replica set deployment for production system is a three-member replica set. These sets provide redundancy and fault tolerance. Avoid complexity when possible, but let your application requirements dictate the architecture.

Replica Set Arbiter

只能把arbiter 增加到偶数个成员的副本集，如果你把arbiter 增加到奇数成员的副本集中，可能造成捆绑选举，造成选择不出primary副本集。

Delayed Replica Set Members

Delayed members contain copies of a replica set’s data set. However, a delayed member’s data set reflects an earlier, or delayed, state of the set. For example, if the current time is 09:52 and a member has a delay of an hour, the delayed member has no operation more recent than 08:52.