MySQL 常用函数积累,提升性能
是否包含线上的点?MBRWithin、MBRContains和java实现包含线上的点,ST_Contains、ST_Within、GISWithin不包含线上的点
查询效率:mysql5.6存储引擎Innodb无法创建空间索引,若无法创建空间索引,使用mysql函数和java哪种查询效率更高?
mysql空间索引:对于空间数据类型的字段建立的索引,MySQL中的空间数据类型有4种,分别是:GEOMETRY,POINT,LINESTRING,POLYGON,MySQL使用SPATIAL关键字进行扩展,使得能够用于创建正规索引类似的语法创建空间索引,创建空间索引的列,必须将其声明为NOT NULL,空间索引只能在存储引擎为MyISAM的表中创建
使用函数时,多边形的第一个点和最后 一个点的gps数据要相同。
方法一:使用mysql函数MBRWithin(A,B) –> A在B中
SELECT
MBRWithin (
GeomFromText ( 'POINT(113.937612 22.534883)' ),
GeomFromText ( 'POLYGON((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488))' )
);
SELECT
MBRWithin (
GeomFromText ( 'POINT(113.971694 22.549804)' ),
GeomFromText ( 'MULTIPOLYGON(((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488)),
((113.972125 22.553275,113.971694 22.549804,113.980461 22.549637,113.978449 22.555345,113.974461 22.556713,113.972125 22.553275)),
((113.971047 22.57033,113.979455 22.563288,113.986139 22.569362,113.981647 22.573167,113.973814 22.573434,113.971047 22.57033)))'
)
);
方法二:使用mysql函数MBRContains(A,B) –> A包含B
SELECT
MBRContains (
GeomFromText ( 'POLYGON((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488))' ),
GeomFromText ( 'POINT(113.980461 22.549637)' )
);
SELECT
MBRContains (
GeomFromText ( 'MULTIPOLYGON(((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488)),
((113.972125 22.553275,113.971694 22.549804,113.980461 22.549637,113.978449 22.555345,113.974461 22.556713,113.972125 22.553275)),
((113.971047 22.57033,113.979455 22.563288,113.986139 22.569362,113.981647 22.573167,113.973814 22.573434,113.971047 22.57033)))' ),
GeomFromText ( 'POINT(113.971694 22.549804)' )
);
方法三:使用mysql函数ST_Contains(A,B) –> A是否包含B
SELECT
ST_Contains (
GeomFromText ( 'POLYGON((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488))' ),
GeomFromText ( 'POINT(113.980461 22.549637)' ) );
SELECT
ST_Contains (
GeomFromText ( 'MULTIPOLYGON(((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488)),
((113.972125 22.553275,113.971694 22.549804,113.980461 22.549637,113.978449 22.555345,113.974461 22.556713,113.972125 22.553275)),
((113.971047 22.57033,113.979455 22.563288,113.986139 22.569362,113.981647 22.573167,113.973814 22.573434,113.971047 22.57033)))' ),
GeomFromText ( 'POINT(113.971694 22.549804)' ) );
方法四:使用mysql函数ST_Within(A,B) –> A是否在B中
SELECT
ST_Within (
GeomFromText ( 'POINT(113.937612 22.534883)' ),
GeomFromText ( 'POLYGON((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488))' ) );
SELECT
ST_Within (
GeomFromText ( 'POINT(113.971694 22.549804)' ),
GeomFromText ( 'MULTIPOLYGON(((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488)),
((113.972125 22.553275,113.971694 22.549804,113.980461 22.549637,113.978449 22.555345,113.974461 22.556713,113.972125 22.553275)),
((113.971047 22.57033,113.979455 22.563288,113.986139 22.569362,113.981647 22.573167,113.973814 22.573434,113.971047 22.57033)))' )
);
方法五:创建函数GISWithin
CREATE FUNCTION `GISWithin`(pt POINT, mp MULTIPOLYGON) RETURNS int(1) DETERMINISTIC
BEGIN
DECLARE str, xy TEXT;
DECLARE x, y, p1x, p1y, p2x, p2y, m, xinters DECIMAL(16, 13) DEFAULT 0;
DECLARE counter INT DEFAULT 0;
DECLARE p, pb, pe INT DEFAULT 0;
SELECT MBRWithin(pt, mp) INTO p;
IF p != 1 OR ISNULL(p) THEN
RETURN p;
END IF;
SELECT X(pt), Y(pt), ASTEXT(mp) INTO x, y, str;
SET str = REPLACE(str, 'POLYGON((','');
SET str = REPLACE(str, '))', '');
SET str = CONCAT(str, ',');
SET pb = 1;
SET pe = LOCATE(',', str);
SET xy = SUBSTRING(str, pb, pe - pb);
SET p = INSTR(xy, ' ');
SET p1x = SUBSTRING(xy, 1, p - 1);
SET p1y = SUBSTRING(xy, p + 1);
SET str = CONCAT(str, xy, ',');
WHILE pe > 0 DO
SET xy = SUBSTRING(str, pb, pe - pb);
SET p = INSTR(xy, ' ');
SET p2x = SUBSTRING(xy, 1, p - 1);
SET p2y = SUBSTRING(xy, p + 1);
IF p1y < p2y THEN SET m = p1y; ELSE SET m = p2y; END IF;
IF y > m THEN
IF p1y > p2y THEN SET m = p1y; ELSE SET m = p2y; END IF;
IF y <= m THEN
IF p1x > p2x THEN SET m = p1x; ELSE SET m = p2x; END IF;
IF x <= m THEN
IF p1y != p2y THEN
SET xinters = (y - p1y) * (p2x - p1x) / (p2y - p1y) + p1x;
END IF;
IF p1x = p2x OR x <= xinters THEN
SET counter = counter + 1;
END IF;
END IF;
END IF;
END IF;
SET p1x = p2x;
SET p1y = p2y;
SET pb = pe + 1;
SET pe = LOCATE(',', str, pb);
END WHILE;
RETURN counter % 2;
END;
# 查询语句如下:
select GISWithin(GeomFromText('POINT(113.980461 22.549637)'),
GeomFromText('POLYGON((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488))'));
select GISWithin(GeomFromText('POINT(113.971694 22.549804)'),GeomFromText('MULTIPOLYGON(((113.942248 22.538488,113.937612 22.534883,113.937756 22.530477,113.954644 22.530811,113.951662 22.538522,113.942248 22.538488)),((113.972125 22.553275,113.971694 22.549804,113.980461 22.549637,113.978449 22.555345,113.974461 22.556713,113.972125 22.553275)),((113.971047 22.57033,113.979455 22.563288,113.986139 22.569362,113.981647 22.573167,113.973814 22.573434,113.971047 22.57033)))'));
方法六:使用java代码判断
方法六:使用java代码判断
PointUtil.java
import java.awt.geom.Point2D;
import java.util.ArrayList;
import java.util.List;
/**
* @author User
* @date 2018-08-22 22:20
* @desc
*/
public class PointUtil {
/**
* 判断点是否在多边形内
*
* @param point 检测点
* @param pts 多边形的顶点
* @return 点在多边形内返回true, 否则返回false
*/
public static boolean IsPtInPoly(Point2D.Double point, List<Point2D.Double> pts) {
int N = pts.size();
boolean boundOrVertex = true; //如果点位于多边形的顶点或边上,也算做点在多边形内,直接返回true
int intersectCount = 0;//cross points count of x
double precision = 2e-10; //浮点类型计算时候与0比较时候的容差
Point2D.Double p1, p2;//neighbour bound vertices
Point2D.Double p = point; //当前点
p1 = pts.get(0);//left vertex
for (int i = 1; i <= N; ++i) {//check all rays
if (p.equals(p1)) {
return boundOrVertex;//p is an vertex
}
p2 = pts.get(i % N);//right vertex
if (p.x < Math.min(p1.x, p2.x) || p.x > Math.max(p1.x, p2.x)) {//ray is outside of our interests
p1 = p2;
continue;//next ray left point
}
if (p.x > Math.min(p1.x, p2.x) && p.x < Math.max(p1.x, p2.x)) {//ray is crossing over by the algorithm (common part of)
if (p.y <= Math.max(p1.y, p2.y)) {//x is before of ray
if (p1.x == p2.x && p.y >= Math.min(p1.y, p2.y)) {//overlies on a horizontal ray
return boundOrVertex;
}
if (p1.y == p2.y) {//ray is vertical
if (p1.y == p.y) {//overlies on a vertical ray
return boundOrVertex;
} else {//before ray
++intersectCount;
}
} else {//cross point on the left side
double xinters = (p.x - p1.x) * (p2.y - p1.y) / (p2.x - p1.x) + p1.y;//cross point of y
if (Math.abs(p.y - xinters) < precision) {//overlies on a ray
return boundOrVertex;
}
if (p.y < xinters) {//before ray
++intersectCount;
}
}
}
} else {//special case when ray is crossing through the vertex
if (p.x == p2.x && p.y <= p2.y) {//p crossing over p2
Point2D.Double p3 = pts.get((i + 1) % N); //next vertex
if (p.x >= Math.min(p1.x, p3.x) && p.x <= Math.max(p1.x, p3.x)) {//p.x lies between p1.x & p3.x
++intersectCount;
} else {
intersectCount += 2;
}
}
}
p1 = p2;//next ray left point
}
if (intersectCount % 2 == 0) {//偶数在多边形外
return false;
} else { //奇数在多边形内
return true;
}
}
}
PointUtilTest.java
import java.awt.geom.Point2D;
import java.util.ArrayList;
import java.util.List;
/**
* @author User
* @date 2018-08-22 22:20
* @desc
*/
public class PointUtilTest {
public static void main(String[] args) {
Point2D.Double point = new Point2D.Double(116.404072, 39.916605);
List<Point2D.Double> pts = new ArrayList<Point2D.Double>();
pts.add(new Point2D.Double(116.395, 39.910));
pts.add(new Point2D.Double(116.394, 39.914));
pts.add(new Point2D.Double(116.403, 39.920));
pts.add(new Point2D.Double(116.402, 39.914));
pts.add(new Point2D.Double(116.410, 39.913));
if(PointUtil.IsPtInPoly(point, pts)){
System.out.println("点在多边形内");
}else{
System.out.println("点在多边形外");
}
}
}
MySQL 常用语句积累,提升性能
场景:基础表连接关系表 …
in后面跟的是小表,exists后面跟的是大表。
select * from tb_emp_bigdata A where A.deptno in (select B.deptno from tb_dept_bigdata B)
场景:按照某一状态和时间降序排列
-- 错误:
select xx from table_name order by status,time desc;
-- 正解:
select xx from table_name order by status desc,time desc;
-- 过滤排序(某一状态排在前面,后面按照其他属性排序)2021.04.26 (万楼汇)
order by (case when relation_type=2 then 1 else 2 end),start_time desc,id desc
SELECT
a.order_id,
r.premises_id,
now( ),
0,
now( ),
2
FROM
(
SELECT
pii.province_id,
pii.city_id,
pii.area_id,
pii.premises_name,
pii.premises_type,
wli.order_id
FROM
workorder_lock_info AS wli
INNER JOIN premises_import_info AS pii ON wli.material_id = pii.material_id
AND pii.isdelete = 0
WHERE
wli.isdelete = 0
AND wli.order_status IN ( 2, 3 )
) AS a
INNER JOIN premises_result_info r ON a.province_id = r.province_id
AND a.city_id = r.city_id
AND a.area_id = r.area_id
AND a.premises_name = r.premises_name
-- 城市ID、区域ID、楼盘名称不为空,楼盘类型可能为空(为空时特殊处理:为空串或者为NULL)
AND ( a.premises_type = r.premises_type OR ( a.premises_type IS NULL AND r.premises_type IS NULL ) )
WHERE
r.isdelete = 0
GROUP BY
a.order_id
String、Hash、List、Set、Zset、BitMaps、Hyperloglogs、Geo、Streams ….
Redis底层基于C语言实现。
底层存储: int、float、String …
编码方式:int(存储8字节长整型)、embstr(SDS格式:存储小于44字节的字符串)、raw(储存大于44字节的字符串)
setnx 与 expire 两个操作分开了
=》分段、while(ture)自旋
如果一个对象有多个值,需要存储,应该怎么办?
跟String一样,String可以做的事,Hash基本都可以做。用来存储多个无序键值对,最大存储(2的32次方-1) 约40亿。
底层存储:zipList(双向链表:KV键值对小于512个 & KV长度小于64byte)
hashtable: (dictionary) hash冲突、rehash、扩容 (数组+ 链表)
结构:Map<Strig,Map<String,Object»
购物车信息
key:用户id, field:商品id, value:商品数量
+1:hincr
-1:hdecr
删除:hincrby key field -1
全选:hgetall
商品数:hlen
存储有序字符串,数据可重复,最大存储数据量约40亿。quicklist (有序双向链表) 数据+链表(与hash存储结构不一样的是内部编码)
lpush/rpush
lpop/rpop
: 用户消息列表、公告列表、活动列表、博客的文章列表等等。
思路:存储所有字段,LRANGE 取出一页内容,按顺序显示。
2. 消息队列: 分布式消息队列(PUB/SUB)
存储String类型的无序集合,最大存储数量约40亿(2的32次方-1);用hashtable(数组+链表) 或intset 存储
# 添加元素(1个或多个)
sadd key a b c d
# 获取所有元素
smembers key
# 统计元素
scard key
# 随机获取元素
srandmember key
# 随机弹出元素 (可用作抽奖)
spop key
# 移除元素
spop key a b
# 查看元素是否存在
sismember key a
抽奖: spop key => 随机弹出
点赞、签到、打卡: sadd key(like:消息id) userId1 userId2 userId3 ....
商品筛选:集合操作:交集
# 商品属性
sadd brand:lenovo 联想小新Pro14
sadd type:small 联想小新Pro14
sadd cpu:intel 联想小新Pro14
...
#商品筛选
sinter brand:lenovo type:small cpu:intel ==> 联想小新Pro14
用户关注、推荐模型 (结构、交集)
sorted set 存储有序元素。每个元素都有个score,按照scocre 从小到大排列,socre相同,按照key 的ASCII码排序。
默认使用ziplist编码。
跳表:
# 添加元素(注意顺序:分值在前)
zadd myzsetkey 10 a 20 b 30 c 40 d
# 获取全部元素
zrange myzsetkey 0 -1 withscores
zrevrange myzsetkey 0 -1 withscores
# 根据分值区间获取值
zrangebyscore myzsetkey 20 40
# 获取元素分值
zscore myzsetkey c
# 根据value 或 分值移除元素
zrem myzsetkey a b
zrem myzsetkey 10
# 统计元素个数
zcard myzsetkey
# 根据分值区间或个数
zcount myzsetkey 10 30
# 分值递增(分值增加2)
zincrby myzsetkey 2 a
# 获取元素排名
zrank myzsetkey b
# 获取倒序
zrev myzsetkey
会动态变化的顺序列表
商品标签:
排行榜:(热榜、热搜)
# 为新闻id为ID1000点击数增加1: zincrby hotnews:22222222 1 ID1000
# 获取今天点击最多的10条数据:zrevrange hotnews:22222222 0 15 withscores
字符串类型上定义的位操作,1个字节有8个二进制表示。bit非常节省空间,可以用来做大数据量统计。在线用户统计
思路:如果用户id,对应一个ASCII码,即数字(且不重复)我们将设置具体的值,0101001..... 可以表示:从0-> 指定位数的最大值。如果某一个用户线下了,我们将对应bit位上的值设置0即可
=》类比:海量不重复数据中,查找数据是否存在。
不太精确的基数统计法,用来统计一个集合中不重复的元数个数,网页的UV;Redis中实现的HyperLogLog,只需要12K就能存储2的64次方个数据。
经纬度相关类型:计算距离等
Redis 是一个高性能的key-value数据库,支持主从同步、集群模式。
穿透(key不存)=》 击穿 =》雪崩(升级版击穿)缓存穿透:key对应的数据源不存在,redis访问不到,直接访问DB,IO频繁甚至崩溃;
缓存击穿:key对应的数据源存在,在redis中失效,大并发过来,直接访问DB,IO频繁甚至崩溃;
缓存雪崩:当缓存服务重启或大量缓存失效,大并发过来,直接访问DB,IO频繁甚至崩溃;
重新选举(哨兵模式)
> 每个从节点,都根据自己对 master 复制数据的 offset,来设置一个选举时间,offset 越大(复制数据越多)的从节点,选举时间越靠前,优先进行选举。
> 所有的 master node 开始 slave 选举投票,给要进行选举的 slave 进行投票,如果大部分 master node (N/2 + 1) 都投票给了某个从节点,那么选举通过,那个从节点可以切换成 master。
MySQL 常用语句积累
-- 常用数据库编码UTF8,特殊表情等需要4字节。 所以需要转换为可支持4个字节的utf8mb4_unicode编码(区分大小写)
Host is blocked because of many connection errors.Unblock with ‘mysqladmin flush-hosts
解决because of many connection errors
DELETE
FROM
student
WHERE
id NOT IN
(
SELECT id FROM
(
SELECT min( id ) AS id
FROM student GROUP BY id_card, `name`
) AS a
);
-- zmm
delete from student a
where a.id !=
(
select min(b.id) from student b where a.name=b.name and a.id_card = b.id_card
) ;
A database index is a data structure that improves the speed of data retrieval operations on a database table at the cost of additional writes and storage space to maintain the index data structure. Indexes are used to quickly locate data without having to search every row in a database table every time a database table is accessed. Indexes can be created using one or more columns of a database table, providing the basis for both rapid random lookups and efficient access of ordered records.
数据库索引,是数据库管理系统(DBMS)中一个排序的数据结构,以便于快速查询、维护数据。
我们在常用MySQL客户端Navicate中可以查看,如下图:
第一列表示索引名称,第二列表示字段名称,第三列为索引类型列;常见的索引类型有普通索引、唯一索引、全文索引。
普通索引(Normal):非唯一索引,一般索引。
唯一索引(Unique): 要求字段不能重复,可能为空。主键索引为特殊的唯一索引,并且不为空。
全文索引(Fulltext):针对比较大的数据,只有文本类型才能创建全文索引,如char、varchar、text。
MySQL 8 : SPAITAL 空间索引:MYSQL中的空间数据类型有4种,分别是GEOMETRY、POINT、LINESTRING、POLYGON。创建空间索引的列,必须将其声明为NOT NULL,空间索引只能在存储引擎为MYISAM的表中创建
Mysql 5.6.51-1-log 在 InnoDB 中支持 BTREE、HASH。
二分查找 -> 二叉查找树 -> 二叉平衡树 ->
有序数组等值查找效率比较高,适用于静态存储数据。如果数据变更,数据移动,index 可能会调整。
?有没有支持链表的二分查找呢