共查询到20条相似文献，搜索用时 31 毫秒
1.
Due to high data volume, massive spatial data requires considerable computing power for real‐time processing. Currently, high performance clusters are the only economically viable solution given the development of multicore technology and computer component cost reduction in recent years. Massive spatial data processing demands heavy I/O operations, however, and should be characterized as a data‐intensive application. Data‐intensive application parallelization strategies, such as decomposition, scheduling and load‐balance, are much different from that of traditional compute‐intensive applications. In this article we introduce a Split‐and‐Merge paradigm for spatial data processing and also propose a robust parallel framework in a cluster environment to support this paradigm. The Split‐and‐Merge paradigm efficiently exploits data parallelism for massive data processing. The proposed framework is based on the open‐source TORQUE project and hosted on a multicore‐enabled Linux cluster. A specific data‐aware scheduling algorithm was designed to exploit data sharing between tasks and decrease the data communication time. Two LiDAR point cloud algorithms, IDW interpolation and Delaunay triangulation, were implemented on the proposed framework to evaluate its efficiency and scalability. Experimental results demonstrate that the system provides efficient performance speedup. 相似文献
2.
Today, many real‐time geospatial applications (e.g. navigation and location‐based services) involve data‐ and/or compute‐intensive geoprocessing tasks where performance is of great importance. Cloud computing, a promising platform with a large pool of storage and computing resources, could be a practical solution for hosting vast amounts of data and for real‐time processing. In this article, we explored the feasibility of using Google App Engine (GAE), the cloud computing technology by Google, for a module in navigation services, called Integrated GNSS (iGNSS) QoS prediction. The objective of this module is to predict quality of iGNSS positioning solutions for prospective routes in advance. iGNSS QoS prediction involves the real‐time computation of large Triangulated Irregular Networks (TINs) generated from LiDAR data. We experimented with the Google App Engine (GAE) and stored a large TIN for two geoprocessing operations (proximity and bounding box) required for iGNSS QoS prediction. The experimental results revealed that while cloud computing can potentially be used for development and deployment of data‐ and/or compute‐intensive geospatial applications, current cloud platforms require improvements and special tools for handling real‐time geoprocessing, such as iGNSS QoS prediction, efficiently. The article also provides a set of general guidelines for future development of real‐time geoprocessing in clouds. 相似文献
3.
随着位置信息在各行各业中的广泛应用,空间大数据得到迅猛发展.空间大数据除具有数据量大的特点之外,还具有复杂性,同时,越来越多的应用对数据的实时性也有较高的要求.传统的GIS软件在承载和处理空间数据时,也面临越来越多的挑战,如难以对复杂多样的空间数据进行一体化存储和管理;传统GIS软件架构和单机处理能力,无法对较大体量(10亿条记录或更大)的空间数据进行分析.本文从分布式存储技术、分布式空间处理计算技术和分布式计算协调技术三个方面阐述如何应对上述问题,并提出了将Spark分布式框架和Su-perMap iObject for Spark空间处理引擎相结合的分布式空间处理计算技术,以及数据库的一体化管理和监控技术,实现对多种数据库如PostgreSQL集群、MongoDB和Elasticsearch的统一管理和监控. 相似文献
4.
为应对陆地观测卫星数据及应用的多样性、存储数据规模的急速增长、数据处理与应用的高时效性、系统高可扩展性、系统资源整合的紧迫性等挑战,结合大数据技术,提出了一种基于Hadoop平台支持大数据处理与分析的陆地观测卫星数据中心架构解决方案,以提高陆地观测卫星数据利用水平与决策支持能力,提升国家空间信息资源应用水平,培育我国陆地观测卫星数据应用新的经济增长点。 相似文献
5.
6.
《International Journal of Digital Earth》2013,6(11):901-917
Interpreting spatial data to derive information is a core task in the field of Geographic Information Science and Technology. A logical step following the collection of data in online repositories is to provide geoprocessing technology for analysing data online. Online geoprocessing technology can be employed for providing a specified set of tools in a theme-specific platform, documenting a model or workflow and making it widely available, automating recurring tasks or offering simple tools to a large user group. This systematic analysis of literature evaluates how much available online geoprocessing tools are being used for answering questions in specific application contexts. An initial set of articles is derived from a keyword-based search in the database Scopus. This set of articles is manually filtered to identify applications of online geoprocessing tools. The analysis of application-related articles shows that virtually all applications require further development of tools. Experts outside the spatial information science field are still underrepresented regarding the use of this technology. The required adaptation of technology for user tasks is identified as major barrier for the wide use of online geoprocessing. Further research needs to assess user tasks and how online geoprocessing can provide the required functions in a user-oriented manner. 相似文献
7.
ABSTRACTReal-time geospatial information is used in various applications such as risk management or alerting services. Especially, the rise of new sensing technologies also increases the demand for processing the data in real time. Today’s spatial data infrastructures, however, do not meet the requirements for real-time geoprocessing. The OpenGIS® Web Processing Service (WPS) is not designed to process real-time workflows. It has some major drawbacks in asynchronous processing and cannot handle (geo) data streams out of the box. In previous papers, we introduced the GeoPipes approach to share spatiotemporal data in real time. We implemented the concept extending the Message Queue and Telemetry Transport (MQTT) protocol by a spatial and temporal dimension, which we call GeoMQTT. In this paper, we demonstrate the integration of the GeoPipes idea in the WPS interface to expose standardized real-time geoprocessing services. The proof of the concept is illustrated in some exemplary real-time geo processes. 相似文献
8.
《International Journal of Digital Earth》2013,6(4):404-425
Abstract The emergence of Cloud Computing technologies brings a new information infrastructure to users. Providing geoprocessing functions in Cloud Computing platforms can bring scalable, on-demand, and cost–effective geoprocessing services to geospatial users. This paper provides a comparative analysis of geoprocessing in Cloud Computing platforms – Microsoft Windows Azure and Google App Engine. The analysis compares differences in the data storage, architecture model, and development environment based on the experience to develop geoprocessing services in the two Cloud Computing platforms; emphasizes the importance of virtualization; recommends applications of hybrid geoprocessing Clouds, and suggests an interoperable solution on geoprocessing Cloud services. The comparison allows one to selectively utilize Cloud Computing platforms or hybrid Cloud pattern, once it is understood that the current development of geoprocessing Cloud services is restricted to specific Cloud Computing platforms with certain kinds of technologies. The performance evaluation is also performed over geoprocessing services deployed in public Cloud platforms. The tested services are developed using geoprocessing algorithms from different vendors, GeoSurf and Java Topology Suite. The evaluation results provide a valuable reference on providing elastic and cost-effective geoprocessing Cloud services. 相似文献
9.
《International Journal of Digital Earth》2013,6(2):111-134
Abstract While significant progress has been made to implement the Digital Earth vision, current implementation only makes it easy to integrate and share spatial data from distributed sources and has limited capabilities to integrate data and models for simulating social and physical processes. To achieve effectiveness of decision-making using Digital Earth for understanding the Earth and its systems, new infrastructures that provide capabilities of computational simulation are needed. This paper proposed a framework of geospatial semantic web-based interoperable spatial decision support systems (SDSSs) to expand capabilities of the currently implemented infrastructure of Digital Earth. Main technologies applied in the framework such as heterogeneous ontology integration, ontology-based catalog service, and web service composition were introduced. We proposed a partition-refinement algorithm for ontology matching and integration, and an algorithm for web service discovery and composition. The proposed interoperable SDSS enables decision-makers to reuse and integrate geospatial data and geoprocessing resources from heterogeneous sources across the Internet. Based on the proposed framework, a prototype to assist in protective boundary delimitation for Lunan Stone Forest conservation was implemented to demonstrate how ontology-based web services and the services-oriented architecture can contribute to the development of interoperable SDSSs in support of Digital Earth for decision-making. 相似文献
10.
11.
《International Journal of Digital Earth》2013,6(9):995-1012
ABSTRACTEarth observation (EO) data, such as high-resolution satellite imagery or LiDAR, has become one primary source for forests Aboveground Biomass (AGB) mapping and estimation. However, managing and analyzing the large amount of globally or locally available EO data remains a great challenge. The Google Earth Engine (GEE), which leverages cloud-computing services to provide powerful capabilities on the management and rapid analysis of various types of EO data, has appeared as an inestimable tool to address this challenge. In this paper, we present a scalable cyberinfrastructure for on-the-fly AGB estimation, statistics, and visualization over a large spatial extent. This cyberinfrastructure integrates state-of-the-art cloud computing applications, including GEE, Fusion Tables, and the Google Cloud Platform (GCP), to establish a scalable, highly extendable, and high-performance analysis environment. Two experiments were designed to demonstrate its superiority in performance over the traditional desktop environment and its scalability in processing complex workflows. In addition, a web portal was developed to integrate the cyberinfrastructure with some visualization tools (e.g. Google Maps, Highcharts) to provide a Graphical User Interfaces (GUI) and online visualization for both general public and geospatial researchers. 相似文献
12.
Big geospatial data is an emerging sub‐area of geographic information science, big data, and cyberinfrastructure. Big geospatial data poses two unique challenges. First, raster and vector data structures and analyses have developed on largely separate paths for the last 20 years. This is creating an impediment to geospatial researchers seeking to utilize big data platforms that do not promote heterogeneous data types. Second, big spatial data repositories have yet to be integrated with big data computation platforms in ways that allow researchers to spatio‐temporally analyze big geospatial datasets. IPUMS‐Terra, a National Science Foundation cyberInfrastructure project, addresses these challenges by providing a unified framework of integrated geospatial services which access, analyze, and transform big heterogeneous spatio‐temporal data. As IPUMS‐Terra's data volume grows, we seek to integrate geospatial platforms that will scale geospatial analyses and address current bottlenecks within our system. However, our work shows that there are still unresolved challenges for big geospatial analysis. The most pertinent is that there is a lack of a unified framework for conducting scalable integrated vector and raster data analysis. We conducted a comparative analysis between PostgreSQL with PostGIS and SciDB and concluded that SciDB is the superior platform for scalable raster zonal analyses. 相似文献
13.
《International Journal of Digital Earth》2013,6(7):737-748
ABSTRACTSupervised image classification has been widely utilized in a variety of remote sensing applications. When large volume of satellite imagery data and aerial photos are increasingly available, high-performance image processing solutions are required to handle large scale of data. This paper introduces how maximum likelihood classification approach is parallelized for implementation on a computer cluster and a graphics processing unit to achieve high performance when processing big imagery data. The solution is scalable and satisfies the need of change detection, object identification, and exploratory analysis on large-scale high-resolution imagery data in remote sensing applications. 相似文献
14.
Spatial cloud computing: how can the geospatial sciences use and help shape cloud computing? 总被引：1，自引：0，他引：1
《International Journal of Digital Earth》2013,6(4):305-329
Abstract The geospatial sciences face grand information technology (IT) challenges in the twenty-first century: data intensity, computing intensity, concurrent access intensity and spatiotemporal intensity. These challenges require the readiness of a computing infrastructure that can: (1) better support discovery, access and utilization of data and data processing so as to relieve scientists and engineers of IT tasks and focus on scientific discoveries; (2) provide real-time IT resources to enable real-time applications, such as emergency response; (3) deal with access spikes; and (4) provide more reliable and scalable service for massive numbers of concurrent users to advance public knowledge. The emergence of cloud computing provides a potential solution with an elastic, on-demand computing platform to integrate – observation systems, parameter extracting algorithms, phenomena simulations, analytical visualization and decision support, and to provide social impact and user feedback – the essential elements of the geospatial sciences. We discuss the utilization of cloud computing to support the intensities of geospatial sciences by reporting from our investigations on how cloud computing could enable the geospatial sciences and how spatiotemporal principles, the kernel of the geospatial sciences, could be utilized to ensure the benefits of cloud computing. Four research examples are presented to analyze how to: (1) search, access and utilize geospatial data; (2) configure computing infrastructure to enable the computability of intensive simulation models; (3) disseminate and utilize research results for massive numbers of concurrent users; and (4) adopt spatiotemporal principles to support spatiotemporal intensive applications. The paper concludes with a discussion of opportunities and challenges for spatial cloud computing (SCC). 相似文献
15.
三维激光扫描直接对地球表面进行三维密集采样,可快速获取具有三维坐标(X,Y,Z)和一定属性(反射强度等)的海量、不规则空间分布三维点云,成为数字化时代下刻画复杂现实世界最为直接和重要的三维地理空间数据获取手段,在全球变化、智慧城市、全球制图等国家重大需求和地球系统科学研究中起到十分重要的作用。目前,在传感器技术和国家需求的双重驱动下,三维激光扫描在硬件装备、三维点云数据处理以及应用3个方面取得了巨大的进步,同时也面临新的挑战。本文以三维激光扫描的发展历史为线索,总结了三维激光扫描系统的现状、三维点云数据处理的关键进展以及在测绘地理信息等领域的典型应用,并分析了三维点云数据处理面临的挑战,最后展望了三维激光扫描与点云处理的发展趋势。 相似文献
16.
17.
波形分解是机载激光雷达全波形数据处理的重要基础工作,通过求解波形函数模型的参数,将波形数据利用具体的函数模型拟合出来,实现对全波形及其中各个子波形函数表达。LM(Levenberg-Marquardt)算法及其改进的算法是波形分解中对参数进行拟合求解的常用方法。针对LM算法在参数拟合计算的过程中存在大量迭代和矩阵运算,提出了基于线程块组和线程两级并行粒度的并行计算方案。将串行多次循环迭代求解参数改为单次并行计算取最佳值实现对参数的选择,将矩阵运算进行线程块的协同并行计算,实现了LM算法在通用计算图形处理器上的并行计算。实验证明,在规定阈值条件下,并行LM降低了算法的迭代次数,提高了波形分解LM算法的计算效率,为提高波形分解的处理效率提供了研究思路。 相似文献
18.
In a service‐oriented environment, Web geoprocessing services can provide geoprocessing functions for a variety of applications including Sensor Web. Connecting Sensor Web and geoprocessing services together shows great potentail to support live geoprocessing using real‐time data inputs. This article proposes a task ontology driven approach to live geoprocessing. The task in the ontology contains five aspects: task type, task priority, task constraints, task model, and task process. The use of the task ontology in driving live geoprocessing includes the following steps: (1) Task model generation, which generates a concrete process model to fulfill user demands; (2) Process model instantiation, which transforms the process model into an executable workflow; (3) Workflow execution: the workflow engine executes the workflow to generate value‐added data products using Sensor Web data as inputs. The approach not only helps create semantically correct connections between Sensor Web and Web geoprocessing services, but also provides sharable problem solving knowledge using process models. A prototype system, which leverages Web 2.0, Sensor Web, Semantic Web, and geoprocessing services, is developed to demonstrate the applicability of the approach. 相似文献
19.
20.
GeoPW: Laying Blocks for the Geospatial Processing Web 总被引：2，自引：0，他引：2
Peng Yue Jianya Gong Liping Di Jie Yuan Lizhi Sun Ziheng Sun Qian Wang 《Transactions in GIS》2010,14(6):755-772
Recent advances in Web‐related technologies have significantly promoted the wide sharing and integrated analysis of distributed geospatial data. Geospatial applications often involve diverse sources of data and complex geoprocessing functions. Existing Web‐based GIS focuses more on access to distributed geospatial data. In scientific problem solving, the ability to carry out geospatial analysis is essential to geoscientific discovery. This article presents the design and implementation of GeoPW, a set of services providing geoprocessing functions over the Web. The concept of the Geospatial Processing Web is discussed to address the geoprocessing demands in the emerging information infrastructure, and the role of GeoPW in establishing the Geospatial Processing Web is identified. The services in GeoPW are implemented by developing middleware that wraps legacy GIS analysis components to provide a large number of geoprocessing utilities over the Web. These services are open and accessible to the public, and they support integrated geoprocessing on the Web. 相似文献