文章目录
1、MySQL集群技术之Galera
之前我写过关于Galera集群的相关文章、关于什么是Galera以及他解决了什么问题;这里我就不再详细介绍了、感兴趣的同学可以自行去查看文章:Galera Cluster
MySQL官方也提供了一个数据库集群方案:MySQL Cluster、但是MySQL Cluster集群技术是有缺点的;例如 MySQL Cluster 不支持 InnoDB 存储引擎、MySQL Cluster 默认必须是 NDBCluster 存储引擎;还有就是 MySQL Cluster 的集群稳定性较差。那么除了官方提供的 MySQL Cluster 集群方案还有其他的集群方案吗?答案是有的、这就是今天我们要介绍的 MySQL Galera 数据库集群方案。 MySQL Galera 是第三方的数据库集群方案、所以我们在部署的时候需要给MySQL打上相应的版本补丁才能把 MySQL Galera 集群跑起来。
MySQL Cluster官方网站:https://dev.mysql.com/downloads/cluster/
Galera官方网站:https://galeracluster.com/downloads/
1.1、下载安装Galera Replication
注:这里我们准备4个节点来部署Galera、宿主机基础信息如下:
[root@mysql-01 ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[root@mysql-01 ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
Galera Replication并不借助于第三方的存储来实现数据库集群、所有的数据都存储在各节点本地;通过 Galera Replication 集群技术各节点都可以参与数据的读和写。Galera Replication 的集群架构如下:
从上面的官方截图我们可以看到、Galera版本和常规的MysQL Server版本相比、多了一个叫做WSREP模块的补丁;我们可以单独去下载Galera、有可以去下载已经打好WSREP模块补丁的MySQl数据库版本。其实上面的Galera链接本身就是一个Yum源(从repodata就可以看出来)、这里我们来把Galera配置为Yum源、详细操作如下:
# 修改Yum源
[root@mysql-01 ~]# cat /etc/yum.repos.d/galera.repo
[galera]
name=galera
baseurl=http://releases.galeracluster.com/mysql-wsrep-5.7/centos/7/x86_64/
gpgcheck=0
[root@mysql-01 ~]#
# 为了方便其他节点安装Galera、我们把Yum源下载的内容缓存下来做成本地Yum源加快安装速度
[root@mysql-01 ~]# cat /etc/yum.conf
[main]
cachedir=/var/cache/yum/$basearch/$releasever
# 开启缓存
keepcache=1
......
# 可以看到galera已经在Yum源里面了
[root@mysql-01 ~]# yum repolist
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
epel/x86_64/metalink | 8.0 kB 00:00:00
* base: mirrors.aliyun.com
* epel: mirrors.ustc.edu.cn
* extras: mirrors.aliyun.com
* updates: mirrors.163.com
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
galera | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
galera/primary_db | 11 kB 00:00:16
repo id repo name status
base/7/x86_64 CentOS-7 - Base 10,072
epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 13,550
extras/7/x86_64 CentOS-7 - Extras 453
galera galera 8
updates/7/x86_64 CentOS-7 - Updates 1,729
repolist: 25,812
[root@mysql-02 ~]#
# galera Yum源包含的包
[root@mysql-01 ~]# yum list | egrep 'wsrep|galera'
galera.x86_64 25.3.12-2.el7 epel
mysql-wsrep-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-client-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-common-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-devel-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-libs-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-libs-compat-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-server-5.7.x86_64 5.7.32-25.24.el7 galera
mysql-wsrep-test-5.7.x86_64 5.7.32-25.24.el7 galera
[root@mysql-01 ~]#
# 执行安装命令
[root@mysql-01 ~]# yum install mysql-wsrep-5.7.x86_64 galera -y
注:这里galera如果安装不上可以添加一下epel源:yum install -y epel-release
1.2、配置本地Yum源
# 把缓存下来的galera包拷贝到galera目录
[root@mysql-01 ~]# mkdir -p galera
[root@mysql-01 ~]# find /var/cache/yum/x86_64/7/ -iname "*.rpm" -exec cp -a {} galera \;
[root@mysql-01 ~]# ls galera/
lsof-4.87-6.el7.x86_64.rpm
mysql-wsrep-common-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-server-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-libs-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-client-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-libs-compat-5.7-5.7.32-25.24.el7.x86_64.rpm
[root@mysql-01 ~]#
# 安装vsftpd和createrepo
[root@mysql-01 ~]# yum install vsftpd createrepo -y
[root@mysql-01 ~]# cp -r galera /var/ftp
[root@mysql-01 ~]# ls /var/ftp/galera/
lsof-4.87-6.el7.x86_64.rpm
mysql-wsrep-common-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-server-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-libs-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-client-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-libs-compat-5.7-5.7.32-25.24.el7.x86_64.rpm
[root@mysql-01 ~]#
# 创建本地Yum仓库
[root@mysql-01 ~]# createrepo /var/ftp/galera/
Spawning worker 0 with 2 pkgs
Spawning worker 1 with 2 pkgs
Spawning worker 2 with 2 pkgs
Spawning worker 3 with 1 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@mysql-01 ~]#
[root@mysql-01 ~]# ls /var/ftp/galera/
lsof-4.87-6.el7.x86_64.rpm
mysql-wsrep-common-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-server-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-libs-5.7-5.7.32-25.24.el7.x86_64.rpm
repodata
mysql-wsrep-client-5.7-5.7.32-25.24.el7.x86_64.rpm
mysql-wsrep-libs-compat-5.7-5.7.32-25.24.el7.x86_64.rpm
[root@mysql-01 ~]#
# 启动sftp、Yum源就已经做好了
[root@mysql-01 ~]# systemctl start vsftpd
[root@mysql-01 ~]# systemctl enable vsftpd
Created symlink from /etc/systemd/system/multi-user.target.wants/vsftpd.service to /usr/lib/systemd/system/vsftpd.service.
[root@mysql-01 ~]#
1.3、使用本地源安装Galera Replication
# 配置本地Yum源
[root@mysql-02 ~]# cat /etc/yum.repos.d/galera.repo
[galera]
name=galera
baseurl=ftp://172.16.200.51/galera
gpgcheck=0
[root@mysql-02 ~]#
# 安装galera
[root@mysql-02 ~]# yum install mysql-wsrep-5.7.x86_64 galera -y
[root@mysql-02 ~]# systemctl start mysqld && systemctl enable mysqld
[root@mysql-02 ~]#
[root@mysql-02 ~]# netstat -nltp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1022/sshd
tcp6 0 0 :::3306 :::* LISTEN 2234/mysqld
tcp6 0 0 :::22 :::* LISTEN 1022/sshd
[root@mysql-02 ~]# systemctl status mysqld
● mysqld.service - MySQL Server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2021-03-10 23:45:45 CST; 15h ago
Docs: man:mysqld(8)
http://dev.mysql.com/doc/refman/en/using-systemd.html
Main PID: 2234 (mysqld)
CGroup: /system.slice/mysqld.service
└─2234 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid --wsrep_start_position=c06cce01-0000-0000-20f7-d60100000000:41
Mar 10 23:45:31 mysql-03 systemd[1]: Starting MySQL Server...
Mar 10 23:45:45 mysql-03 systemd[1]: Started MySQL Server.
[root@mysql-02 ~]#
# 查看默认密码
[root@mysql-02 ~]# grep 'password' /var/log/mysqld.log
2021-03-10T15:45:35.571878Z 1 [Note] A temporary password is generated for root@localhost: gYw!-TbtQ0N.
2021-03-10T15:45:43.637647Z 0 [Note] Shutting down plugin 'validate_password'
2021-03-10T15:45:45.259818Z 0 [Note] Shutting down plugin 'sha256_password'
2021-03-10T15:45:45.259827Z 0 [Note] Shutting down plugin 'mysql_native_password'
2021-03-11T01:16:11.979808Z 2 [Note] Access denied for user 'root'@'localhost' (using password: NO)
# 修改默认密码
[root@mysql-02 ~]# newpass=`grep 'temporary password' /var/log/mysqld.log | awk '{print $NF}'`;mysqladmin -p"$newpass" password 'QWEqwe123!@#'
注:其他节点按照同样的操作步骤操作;这样我们就把所有节点的Galera安装好了、下面我们开始配置Galera。
1.4、配置Galera Replication
galera安装完成以后我们还需要创建一个用于数据同步的用户、如下:
# 创建一个名为sst的用户并授权访问所有库和所有表
[root@mysql-01 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.7.32 MySQL Community Server - (GPL), wsrep_25.24
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> GRANT ALL ON *.* TO 'sst'@'172.16.200.%' IDENTIFIED BY 'QWEqwe123!@#';
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql>
注:在所有节点上都要操作上面的命令。
数据同步用户创建完成之后我们就可以来配置Galera了。Galera的主要配置文件我们放在 /etc/my.cnf 下面、节点 mysql-01 的具体内容如下:
[root@mysql-01 ~]# cat /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
# 集群内唯一,这里我用的是IP地址最后一位数字
server-id=50
# binlog日志格式
binlog_format=row
# 存储引擎
default_storage_engine=InnoDB
# 使用独立表空间
innodb_file_per_table=1
innodb_autoinc_lock_mode=2
# Galera相关配置
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
# 集群名字,可以自定义,但是同一集群要保持一致
wsrep_cluster_name='galera cluster'
# 集群通讯地址
wsrep_cluster_address='gcomm://'
wsrep_node_name='mysql-01'
wsrep_node_address=172.16.200.50
wsrep_sst_auth=sst:QWEqwe123!@#
wsrep_sst_method=rsync
[root@mysql-01 ~]#
特别说明:”gcomm://” 是特殊的地址, wsrep_cluster_address 初始值要配置为 ‘gcomm://’ ,因为初始配置的时候是没有集群存在的;当mysql-01节点启动以后才有集群。启动其他节点就可以添加到mysql-01所在的 galera cluster 集群了;其他节点要配置为集群除自身外所有节点IP地址。当 mysql-01 节点需要重启的时候、我们需要把配置信息修改为:wsrep_cluster_address=’gcomm://172.16.200.51,172.16.200.52,172.16.200.53’。
1.5、Galera集群验证
# Galera验证
[root@mysql-01 ~]# netstat -nltp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1033/sshd
tcp 0 0 0.0.0.0:4567 0.0.0.0:* LISTEN 25186/mysqld
tcp6 0 0 :::3306 :::* LISTEN 25186/mysqld
tcp6 0 0 :::22 :::* LISTEN 1033/sshd
[root@mysql-01 ~]#
[root@mysql-01 ~]# mysql -u root -p
[root@mysql-01 ~]# mysql> show status like 'wsrep%';
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 4
Current database: *** NONE ***
+------------------------------+--------------------------------------+
| Variable_name | Value |
+------------------------------+--------------------------------------+
| wsrep_local_state_uuid | 79bb6036-8242-11eb-8317-7e08328e1e68 |
......
| wsrep_incoming_addresses | 172.16.200.50:3306 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0/0/0/0/0 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_gcomm_uuid | 79bab1cc-8242-11eb-9f50-bed0c9cad008 |
| wsrep_cluster_conf_id | 1 |
| wsrep_cluster_size | 1 |
| wsrep_cluster_state_uuid | 79bb6036-8242-11eb-8317-7e08328e1e68 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 0 |
| wsrep_provider_name | Galera |
......
| wsrep_ready | ON |
+------------------------------+--------------------------------------+
56 rows in set (0.01 sec)
[root@mysql-01 ~]# mysql>
从上面的配置信息我们可以看到mysql已经正常启动并监听3306端口、同时也开启了节点通信端口4567;进入数据库执行 show status like ‘wsrep%’; 命令也可以查询 wsrep 信息也可以看到 wsrep_cluster_size 数量为 1;wsrep_connected=ON、wsrep_ready=ON 则说明节点成功接入集群。
1.6、其他节点加入集群并验证
节点 mysql-02、mysql-03、mysql-04 的配置文件如下:
# mysql-02配置信息如下:
server-id=51
binlog_format=row
default_storage_engine=InnoDB
innodb_file_per_table=1
innodb_autoinc_lock_mode=2
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_name='galera cluster'
wsrep_cluster_address='gcomm://172.16.200.50,172.16.200.52,172.16.200.53'
wsrep_node_name='mysql-02'
wsrep_node_address=172.16.200.51
wsrep_sst_auth=sst:QWEqwe123!@#
wsrep_sst_method=rsync
# mysql-03配置信息如下:
server-id=52
binlog_format=row
default_storage_engine=InnoDB
innodb_file_per_table=1
innodb_autoinc_lock_mode=2
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_name='galera cluster'
wsrep_cluster_address='gcomm://172.16.200.50,172.16.200.51,172.16.200.53'
wsrep_node_name='mysql-03'
wsrep_node_address=172.16.200.52
wsrep_sst_auth=sst:QWEqwe123!@#
wsrep_sst_method=rsync
# mysql-04配置信息如下:
server-id=53
binlog_format=row
default_storage_engine=InnoDB
innodb_file_per_table=1
innodb_autoinc_lock_mode=2
wsrep_on=ON
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_name='galera cluster'
wsrep_cluster_address='gcomm://172.16.200.50,172.16.200.51,172.16.200.52'
wsrep_node_name='mysql-04'
wsrep_node_address=172.16.200.53
wsrep_sst_auth=sst:QWEqwe123!@#
wsrep_sst_method=rsync
配置修改完成以后我们启动所有节点的 mysql 数据库、启动正常我们就可以到集群里面来验证一下所有节点是否都已经加入到集群中了。这里再重复提醒一下、mysql-02、mysql-03、mysql-04节点启动以后必须把 mysql-01 节点的配置信息修改为:wsrep_cluster_address=’gcomm://172.16.200.51,172.16.200.52,172.16.200.53’、然后重启 mysql-01节点。
mysql-01节点重启以后我们进入数据库来验证一下集群信息:
[root@mysql-01 ~]# mysql -u root -p
[root@mysql-01 ~]# mysql> show status like 'wsrep%';
+------------------------------+-----------------------------------------------------------------------------+
| Variable_name | Value |
+------------------------------+-----------------------------------------------------------------------------+
| wsrep_local_state_uuid | 79bb6036-8242-11eb-8317-7e08328e1e68 |
| wsrep_protocol_version | 7 |
| wsrep_last_committed | 0 |
| wsrep_replicated | 0 |
| wsrep_replicated_bytes | 0 |
| wsrep_repl_keys | 0 |
| wsrep_repl_keys_bytes | 0 |
| wsrep_repl_data_bytes | 0 |
| wsrep_repl_other_bytes | 0 |
| wsrep_received | 15 |
| wsrep_received_bytes | 2256 |
| wsrep_local_commits | 0 |
| wsrep_local_cert_failures | 0 |
| wsrep_local_replays | 0 |
| wsrep_local_send_queue | 0 |
| wsrep_local_send_queue_max | 1 |
| wsrep_local_send_queue_min | 0 |
| wsrep_local_send_queue_avg | 0.000000 |
| wsrep_local_recv_queue | 0 |
| wsrep_local_recv_queue_max | 1 |
| wsrep_local_recv_queue_min | 0 |
| wsrep_local_recv_queue_avg | 0.000000 |
| wsrep_local_cached_downto | 18446744073709551615 |
| wsrep_flow_control_paused_ns | 0 |
| wsrep_flow_control_paused | 0.000000 |
| wsrep_flow_control_sent | 0 |
| wsrep_flow_control_recv | 0 |
| wsrep_cert_deps_distance | 0.000000 |
| wsrep_apply_oooe | 0.000000 |
| wsrep_apply_oool | 0.000000 |
| wsrep_apply_window | 0.000000 |
| wsrep_commit_oooe | 0.000000 |
| wsrep_commit_oool | 0.000000 |
| wsrep_commit_window | 0.000000 |
| wsrep_local_state | 4 |
| wsrep_local_state_comment | Synced |
| wsrep_cert_index_size | 0 |
| wsrep_causal_reads | 0 |
| wsrep_cert_interval | 0.000000 |
| wsrep_incoming_addresses | 172.16.200.50:3306,172.16.200.51:3306,172.16.200.53:3306,172.16.200.52:3306 |
| wsrep_evs_delayed | |
| wsrep_evs_evict_list | |
| wsrep_evs_repl_latency | 0.00166563/0.00216561/0.00253249/0.000336227/4 |
| wsrep_evs_state | OPERATIONAL |
| wsrep_gcomm_uuid | 79bab1cc-8242-11eb-9f50-bed0c9cad008 |
| wsrep_cluster_conf_id | 8 |
| wsrep_cluster_size | 4 |
| wsrep_cluster_state_uuid | 79bb6036-8242-11eb-8317-7e08328e1e68 |
| wsrep_cluster_status | Primary |
| wsrep_connected | ON |
| wsrep_local_bf_aborts | 0 |
| wsrep_local_index | 0 |
| wsrep_provider_name | Galera |
| wsrep_provider_vendor | Codership Oy <info@codership.com> |
| wsrep_provider_version | 3.12(r9921e73) |
| wsrep_ready | ON |
+------------------------------+-----------------------------------------------------------------------------+
56 rows in set (0.00 sec)
[root@mysql-01 ~]# mysql>
注:我们可以看到当前 Galera Cluster 集群节点数量为 4;集群已经正常启动且所有节点都顺利加入到集群中。
我们去 mysql-01 节点创建一个数据库并导入一些测试数据表、然后到其他节点去验证一下:
Last login: Thu Mar 11 15:27:01 2021 from 10.10.1.236
[root@mysql-02 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.7.32 MySQL Community Server - (GPL), wsrep_25.24
Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
| z0ukun |
+--------------------+
5 rows in set (0.01 sec)
mysql> use z0ukun;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+-------------------------------------+
| Tables_in_z0ukun |
+-------------------------------------+
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| chpa_data_city |
| chpa_data_country |
| chpa_data_shirtsales |
| django_admin_log |
| django_celery_beat_clockedschedule |
| django_celery_beat_crontabschedule |
| django_celery_beat_intervalschedule |
| django_celery_beat_periodictask |
| django_celery_beat_periodictasks |
| django_celery_beat_solarschedule |
| django_content_type |
| django_migrations |
| django_session |
+-------------------------------------+
19 rows in set (0.01 sec)
mysql>
问题:这里我们虽然部署好了Galera Cluster 集群、其实终端用户可以连接上面4个节点中的任意一个节点;那么终端用户到底去连接哪个节点呢?下一篇文章我们就来讲解这个内容。
That is very fascinating, You are an excessively
skilled blogger. I’ve joined your feed and sit up for
in the hunt for more of your magnificent post. Also, I’ve
shared your site in my social networks
I’m truly enjoying the design and layout of your website.
It’s a very easy on the eyes which makes it much more enjoyable for me to come here and visit
more often. Did you hire out a developer to create your theme?
Great work!
I am not sure where you are getting your information, but great
topic. I needs to spend some time learning much more or understanding more.
Thanks for fantastic information I was looking for this information for
my mission.
Hello there, You have done a fantastic job.
I’ll definitely digg it and personally suggest to my friends.
I’m sure they will be benefited from this
website.
Hi! This is my first visit to your blog! We are a team of volunteers and
starting a new initiative in a community in the
same niche. Your blog provided us beneficial information to work on. You have done a outstanding job!
Wow that was strange. I just wrote an incredibly long comment but after I clicked submit my comment didn’t
appear. Grrrr… well I’m not writing all that over again. Anyway, just wanted
to say excellent blog!
bookmarked!!, I really like your site!
Very energetic post, I loved that a lot. Will there be a part 2?
Great post however , I was wanting to know if
you could write a litte more on this subject? I’d be very thankful if you could elaborate a little bit more.
Appreciate it!
Aw, this was an extremely good post. Taking the
time and actual effort to create a great article… but
what can I say… I put things off a whole lot and don’t seem to
get nearly anything done.
Thanks for sharing your thoughts. I really appreciate your efforts and
I am waiting for your next write ups thank you once again.
Hey there! Do you know if they make any plugins to assist
with SEO? I’m trying to get my blog to rank for some
targeted keywords but I’m not seeing very good success.
If you know of any please share. Cheers!
You actually make it seem so easy with your presentation but I find
this topic to be really something that I think I would never understand.
It seems too complicated and very broad for me.
I’m looking forward for your next post, I’ll try to get the hang of
it!
I’m not that much of a internet reader to be honest but
your sites really nice, keep it up! I’ll go ahead
and bookmark your website to come back in the future. All the best
This paragraph is genuinely a fastidious one it assists new net users, who
are wishing in favor of blogging.
Hello I am so grateful I found your weblog, I really found you by accident, while I was browsing on Bing for something else, Anyways
I am here now and would just like to say thanks for a incredible post and a
all round entertaining blog (I also love the theme/design), I don’t have time to
read through it all at the moment but I have saved it and also added in your
RSS feeds, so when I have time I will be back to read more,
Please do keep up the great work.
Greate post. Keep writing such kind of info on your site.
Im really impressed by your blog.
Hello there, You have done an excellent job. I’ll certainly digg
it and for my part suggest to my friends. I’m confident they
will be benefited from this site.
Neat blog! Is your theme custom made or did
you download it from somewhere? A design like
yours with a few simple tweeks would really make my blog stand out.
Please let me know where you got your theme. Thanks
After looking over a few of the blog articles on your blog, I really like your
way of writing a blog. I book-marked it to my bookmark site list
and will be checking back soon. Please check
out my web site as well and tell me what you think.
This excellent website really has all of the information I needed about this subject and didn’t know who to ask.
Aw, this was a very nice post. Finding the time and actual effort to make a
superb article… but what can I say… I put things
off a lot and never seem to get nearly anything done.