基于云计算系统的空间查询处理方法     

陈逸然  黄舟 《测绘科学》2016,41(12):273-278

针对传统的关系型空间数据库已经不能很好地适应于超大规模高并发空间查询访问的处理需要的问题,该文着眼于解决大数据时代下地理信息服务所面临的日益严峻的大规模空间查询访问需求,探索了一套基于Spark架构的空间查询实现技术,并给出相应的解决方案。提出一个基于Spark并提供类SQL访问接口的空间查询实现模型GeoSpark SQL,解决了以下关键问题:数据的外包矩形数据生成和标准地理信息数据对Spark的导入导出方法;Spark空间查询算子实现方法;Spark空间索引与查询优化方法。GeoSpark SQL模型在初步实验中,已可以满足实时性的要求,对复杂的空间查询也能有良好的性能表现。  相似文献   

格网邻域滤波在车载激光点云道路边线提取中的方法研究     

田茂义  李鹏飞  俞家勇  马腾 《测绘与空间地理信息》2016,(5)

结合城市车载激光扫描点云道路数据与地物点各自的特点和空间分布,用路面点云控制道路两侧地物点云进行滤波,从而实现道路路面点和地物点的分离,并根据城市道路中含有高出路面边缘的路缘石的结构特性设置阈值来提取路缘石,通过设置合适的格网及其邻域格网中点的密度特性进一步滤波路缘石,生成道路边线。  相似文献   

Enabling point pattern analysis on spatial big data using cloud computing: optimizing and accelerating Ripley’s K function     

Guiming Zhang  Qunying Huang  John H. Keel 《International journal of geographical information science》2016,30(11):2230-2252

Performing point pattern analysis using Ripley’s K function on point events of large size is computationally intensive as it involves massive point-wise comparisons, time-consuming edge effect correction weights calculation, and a large number of simulations. This article presented two strategies to optimize the algorithm for point pattern analysis using Ripley’s K function and utilized cloud computing to further accelerate the optimized algorithm. The first optimization sorted the points on their x and y coordinates and thus narrowed the scope of searching for neighboring points down to a rectangular area around each point in estimating K function. Using the actual study area in computing edge effect correction weights is essential to estimate an unbiased K function, but is very computationally intensive if the study area is of complex shape. The second optimization reused the previously computed weights to avoid repeating expensive weights calculation. The optimized algorithm was then parallelized using Open Multi-Processing (OpenMP) and hybrid Message Passing Interface (MPI)/OpenMP on the cloud computing platform. Performance testing showed that the optimizations effectively accelerated point pattern analysis using K function by a factor of 8 using both the sequential version and the OpenMP-parallel version of the optimized algorithm. While the OpenMP-based parallelization achieved good scalability with respect to the number of CPU cores utilized and the problem size, the hybrid MPI/OpenMP-based parallelization significantly shortened the time for estimating K function and performing simulations by utilizing computing resources on multiple computing nodes. Computational challenge imposed by point pattern analysis tasks on point events of large size involving a large number of simulations can be addressed by utilizing elastic, distributed cloud resources.  相似文献   

三维激光扫描技术在地质实习中的应用研究     

胡超伦 《测绘与空间地理信息》2017,(8)

三维激光扫描技术相对于传统的数据采集方式具有更高精度、远距离获取等优势。本文介绍了三维激光扫描技术的工作原理,并结合地质剖面分析了点云数据的采集以及获取的流程,结合生成的地质剖面模型讨论了点云数据在地质剖面分析方面的优势以及存在的问题。  相似文献   

基于散乱点云特征提取算法研究     

刘春晓  邵为真  邵帅  井文胜 《测绘与空间地理信息》2017,(8)

三维激光扫描测量技术是当前测绘领域研究的热点。点云各个要素特征提取在三维建模中是非常重要的一个环节,也逐渐成为三维点云数据处理中的一个研究重点。目前对点云数据特征提取仍然存在一些问题,大多数算法的研究是针对栅格点云数据进行的,存在容易受到噪声数据的影响以及适应性不强等问题。本文在研究了现有的点云数据特征提取的方法的基础上,将法线差分算法应用到场景内地物提取中,从而实现场景中地物的特征提取。  相似文献   

面向持续交付的空间大数据基础设施设计与实现     

李闻昊  李少华  李绍俊  侯本洲 《测绘与空间地理信息》2017,40(Z1)

随着空间大数据技术的快速发展,越来越多与空间大数据云平台相关的项目逐步规划和建立.GIS建设项目已经朝着平台化、虚拟化和智能化的方向转变.新一代GIS平台建设的重要原则之一,就是要把空间大数据作为企业的重要资产.它需要系统地管理、持续地维护、安全地控制以及深入地挖掘.在持续探索数据价值的过程中,对平台基础设施提出了新的技术要求,包括集成多种开源技术、支持快速的迭代开发、支持持续运行的数据分析任务等.本文研究的重点是结合DevOps等先进的面向持续交付的管理理念,利用相关技术提出空间大数据基础设施的设计方案,使其具备持续交付能力.  相似文献   

地基可见光全天空云图云量图像处理识别方法     

陈青青  李彪  汤志亚  杨玲  王耀萱 《气象科技》2017,45(6):1006-1010

为增强地基可见光全天空云图中云与天空的特征和区别,提高云检测率,基于图像复原和图像增强技术提出一种改善云图质量的方法。该方法采用暗通道去雾算法进行图像复原;采用亮度直方图均衡增强图像纹理细节;综合两种方法,先图像复原,再图像增强。按低能见度薄云、低能见度厚云、高能见度薄云、高能见度厚云4种情况分别进行讨论,结果表明:除高能见度薄云采用单一的图像复原使云检测效果降低外,图像复原和图像增强都能使云检测和云量识别准确率提高;综合二者,云检测和云量识别准确率进一步提高;该方法对薄云和低能见度云图的改善最为显著。  相似文献   

气象数据云数据存储技术及应用     

杨明  陈晔峰  陈晴  肖云  高祝宇  曾悠 《气象科技》2017,45(6):1017-1021

针对气象数据种类多、信息量大、精度高等特点,传统的关系型数据库系统在存储处理、数据读取等方面存在负载饱和、读写性能不理想等问题,利用云数据存储技术,结合气象数据特征,设计了基于云数据存储平台的气象数据存储和处理方法。通过分析气象数据的结构化、半/非结构化特征,采用云关系型数据库存储结构气象数据;采用NoSQL对象库存储半/非结构化气象数据;在数据的存储过程中,应用了气象数据分块压缩技术,进行气象数据存储和传输。结果表明,所讨论的方法具有很好的存储传输效率,能够满足大规模气象数据在业务应用中对存储查询和处理速度的要求。  相似文献   

Aircraft Measurements of Cloud–Aerosol Interaction over East Inner Mongolia   总被引：1，自引：0，他引：1  

Yuhuan LÜ  Hengchi LEI  Jiefan YANG 《大气科学进展》2017,34(8):983-992

To investigate the potential effects of aerosols on the microphysical properties of warm clouds, airborne observational data collected from 2009 to 2011 in Tongliao, Inner Mongolia, China, were statistically analyzed in this study. The results demonstrated that the vertical distribution of the aerosol number concentration(N_a) was similar to that of the clean rural continent. The average aerosol effective diameter(D_e) was maintained at approximately 0.4 μm at all levels. The data obtained during cloud penetrations showed that there was a progressive increase in the cloud droplet concentration(N_c) and liquid water content(LWC) from outside to inside the clouds, while the Nawas negatively related to the Ncand LWC at the same height. The fluctuation of the N_a, Ncand LWC during cloud penetration was more obvious under polluted conditions(Type 1) than under clean conditions(Type 2). Moreover, the wet scavenging of cloud droplets had a significant impact on the accumulation mode of aerosols, especially on particles with diameters less than 0.4 μm. The minimum wet scavenging coefficient within the cloud was close to 0.02 under Type 1 conditions, while it increased to 0.1 under Type 2 conditions,which proved that the cloud wet scavenging effect under Type 1 conditions was stronger than that under Type 2 conditions.Additionally, cloud droplet spectra under Type 1 conditions were narrower, and their horizontal distributions were more homogeneous than those under Type 2 conditions.  相似文献   

Cloud Radiative Effects and Changes Simulated by the Coupled Model Intercomparison Project Phase 5 Models     

Sun-Hee SHIN  Ok-Yeon KIM  Dongmin KIM  Myong-In LEE 《大气科学进展》2017,34(7):859-876

Using 32 CMIP5(Coupled Model Intercomparison Project Phase 5) models, this study examines the veracity in the simulation of cloud amount and their radiative effects(CREs) in the historical run driven by observed external radiative forcing for 1850–2005, and their future changes in the RCP(Representative Concentration Pathway) 4.5 scenario runs for2006–2100. Validation metrics for the historical run are designed to examine the accuracy in the representation of spatial patterns for climatological mean, and annual and interannual variations of clouds and CREs. The models show large spread in the simulation of cloud amounts, specifically in the low cloud amount. The observed relationship between cloud amount and the controlling large-scale environment are also reproduced diversely by various models. Based on the validation metrics,four models—ACCESS1.0, ACCESS1.3, Had GEM2-CC, and Had GEM2-ES—are selected as best models, and the average of the four models performs more skillfully than the multimodel ensemble average.All models project global-mean SST warming at the increase of the greenhouse gases, but the magnitude varies across the simulations between 1 and 2 K, which is largely attributable to the difference in the change of cloud amount and distribution. The models that simulate more SST warming show a greater increase in the net CRE due to reduced low cloud and increased incoming shortwave radiation, particularly over the regions of marine boundary layer in the subtropics. Selected best-performing models project a significant reduction in global-mean cloud amount of about-0.99% K~(-1)and net radiative warming of 0.46 W m~(-2)K~(-1), suggesting a role of positive feedback to global warming.  相似文献