共查询到20条相似文献，搜索用时 62 毫秒
1.
基于BP神经网络的气象格点数据无损压缩方法 总被引：2，自引：0，他引：2
格点资料是目前气象数据存储、传输和应用的主要形式,爆炸性增长的资料给数据的存储、传输带来了巨大压力,设计适合格点资料特点的压缩方案越来越重要。大气是一种连续介质,格点资料是反映其动力学及热力学性质的连续物理量的采样和量化,相邻格点间存在很大的相关性。在二维线性预测基础上引入BP神经网络,建立了基于神经网络的二次预测模型,有效剔除气象格点数据的冗余信息,结合熵编码,提出了一种高效无损压缩新方案。该方案具有极高的压缩比,并能保证在有效精度内数据完全无损。与广泛应用的netCDF格式压缩效率对比试验表明,该方案的编码效率是netCDF格式的3～5倍,接近压缩的理论下限,能极大地减少存储空间及传输时间,适合大气以及地球科学中海量数据的存储和传输,有很好的实际应用前景。 相似文献
2.
基于统计模型的气象数据无损压缩新方法 总被引：4，自引：1，他引：4
对当前广泛使用的气象格点数据结构进行了统计分析，通过分析常用气象要素格点资料相邻格点之间的相关性及计算要素场的符号熵和信息冗余度，认为气象格点数据中存在着明显的信息冗余，具有很高的可压缩性，且相关性越好，可压缩性越高。在此分析基础上建立了气象格点数据的二维线性预测统计模型，剔除冗余信息，并结合Huffman编码，提出了一种气象格点数据无损压缩新方法。该方法可极大提高气象格点数据的压缩率，且能保证在有效精度内数据完全无损。最后对常用气象格点资料进行了压缩对比试验，结果表明，该方案压缩效果明显优于当前国际通用的气象数据压缩编码格式(如GRIB和netCDF码)，从而能够大大提高气象以及地球科学中海量数据采集、存储和传输交换的业务应用效率。 相似文献
3.
自然灾害测报网格及其实现的关键技术研究 总被引：1，自引：0，他引：1
自然灾害测报是一门综合的科学,它涉及到的灾害数据采集、信息资源分布、异构等问题制约了行业应用和信息化发展,而网格是新一代高性能计算环境和信息服务的基础设施,能够实现动态跨地域的资源共享和整合工作。自然灾害测报网格（NDPG）基于网格技术,为灾害测报工作服务。本文对自然灾害测报网格的概念、研究内容和框架体系进行了初步探讨,并讨论了NDPG在实现中的几个关键技术问题。 相似文献
4.
Andrés García García Carolina Perpiñá Carlos de Alfonso Vicente Hernández 《Earth Science Informatics》2013,6(4):199-207
This paper presents the integration of desktop grid infrastructure with GIS technologies, by proposing a parallel resolution method in a generic distributed environment. A case study focused on a discrete facility location problem, in the biomass area, exemplifies the high amount of computing resources (CPU, memory, HDD) required to solve the spatial problem. A comprehensive analysis is undertaken in order to analyse the behaviour of the grid-enabled GIS system. This analysis, consisting of a set of the experiments on the case study, concludes that the desktop grid infrastructure is able to use a commercial GIS system to solve the spatial problem achieving high speedup and computational resource utilization. Particularly, the results of the experiments showed an increase in speedup of fourteen times using sixteen computers and a computational efficiency greater than 87 % compared with the sequential procedure. 相似文献
5.
Space Physics Interactive Data Resource—SPIDR 总被引：1，自引：0，他引：1
Mikhail Zhizhin Eric Kihn Rob Redmon Dmitry Medvedev Dmitry Mishin 《Earth Science Informatics》2008,1(2):79-91
SPIDR (Space Physics Interactive Data Resource) is a standard data source for solar-terrestrial physics, functioning within
the framework of the ICSU World Data Centers. It is a distributed database and application server network, built to select,
visualize and model historical space weather data distributed across the Internet. SPIDR can work as a fully-functional web-application
(portal) or as a grid of web-services, providing functions for other applications to access its data holdings.
相似文献
| Mikhail ZhizhinEmail: |
6.
7.
A. Niazalizadeh Moghadam R. Ravanmehr 《International Journal of Environmental Science and Technology》2018,15(1):149-158
Today’s organizations, industries and research centers are geographically distributed in different sites. To achieve true knowledge of business, mining massive amounts of data is necessary. In earth-related sciences such as meteorology, the date obtained from the various types of sensors is huge because of the high-frequency rate of data acquisition process and also the geographical distribution of weather stations. Therefore, the data mining and knowledge discovery process of this big and distributed data is a challenging work. In this paper, we propose a new distributed data mining approach called multi-agent hierarchical data mining to classify meteorology data, which has been collected from different sites widely distributed around the country (Iran). Our method utilizes a modified version of REPTree algorithm, which has been optimized to work in multi-agent system. To evaluate the proposed approach, it is implemented on 20 million of meteorology data record. Experimental results show that multi-agent hierarchical data mining approach can achieve significant performance improvement over centralized and parallel methods for knowledge discovery in large amounts of data. 相似文献
8.
Hector A. Duran-Limon Jesus Flores-Contreras Nikos Parlavantzas Ming Zhao Angel Meulenert-Peña 《Earth Science Informatics》2016,9(3):365-382
There are many scientific applications that have high performance computing (HPC) demands. Such demands are traditionally supported by cluster- or Grid-based systems. Cloud computing, which has experienced a tremendous growth, emerged as an approach to provide on-demand access to computing resources. The cloud computing paradigm offers a number of advantages over other distributed platforms. For example, the access to resources is flexible and cost-effective since it is not necessary to invest a large amount of money on a computing infrastructure nor pay salaries for maintenance functions. Therefore, the possibility of using cloud computing for running high performance computing applications is attractive. However, it has been shown elsewhere that current cloud computing platforms are not suitable for running some of these kinds of applications since the performance offered is very poor. The reason is mainly the overhead from virtualisation which is extensively used by most cloud computing platforms as a means to optimise resource usage. Furthermore, running HPC applications in current cloud platforms is a complex task that in many cases requires configuring a cluster of virtual machines (VMs). In this paper, we present a lightweight virtualisation approach for efficiently running the Weather Research and Forecasting (WRF) model (a computing- and communication-intensive application) in a cloud computing environment. Our approach also provides a higher-level programming model that automates the process of configuring a cluster of VMs. We assume such a cloud environment can be shared with other types of HPC applications such as mpiBLAST (an embarrassingly parallel application), and MiniFE (a memory-intensive application). Our experimental results show that lightweight virtualisation imposes about 5 % overhead and it substantially outperforms traditional heavyweight virtualisation such as KVM. 相似文献
9.
基于曙光3000计算环境的寒旱区资源环境数据平台建设 总被引：2，自引：0，他引：2
通过对寒区旱区资源环境研究中已积累的大量特色数据、应用模式和程序的分析,对分布式网络环境中的数据采集与管理、应用模式和程序的移植集成、应用程序与数据的交互访问等技术进行了讨论,提出建立由数据中心、中间件服务、应用服务、一站式管理组成的数据平台是提高数据利用效率,方便实现数据与应用集成,提高科研效率的一种有效途径之一.对数据平台建设的主要思路及实现技术进行了详细讨论,并在实验基础上,初步建立了基于曙光3000高性能计算环境的寒区旱区资源环境数据平台,为以后的数据网格建设做了一定的技术准备. 相似文献
10.
We describe a framework designed to simplify the acquisition and integration of data from multiple, diversely formatted, geographically distributed science data sets. Our domain is Heliophysics where measurements of magnetic fields, plasmas, and charged particles are often made in-situ, with the data made available in relatively low volume data sets consisting of time series tables. Data format diversity has proven to be a significant barrier to the type of integrated, multi-mission analysis that is now very important in Heliophysics. Therefore we have developed a Java framework capable of reading, interpreting, and providing uniform access to the science content of any distributed time series data set. The framework exposes data only through fully abstract interfaces that represent data content while hiding all access details such as file format, data file granularity and access protocols. Furthermore, specialized interfaces for representing measurement-specific details are also employed, so that our framework enables data sets to be recast into scientifically interoperable representations. The context of our efforts is an increasingly distributed Heliophysics data environment that employs a collection of discipline-specific Virtual Observatories (VOs), each providing data search and retrieval services for one Heliophysics sub-discipline. Our framework is bundled in a library that ultimately will serve as a universal reader for Heliophysics data, solving the formats problem and serving as key infrastructure for advanced, science-sensitive data manipulation services. 相似文献
11.
卫星气象产品、气候模式预测数据通常与地面观测数据存在偏差,为保证数据的可靠性和合理性,需要对其进行偏差校正,但偏差校正过程往往受具体区域气象特征、方法本身假定等因素的影响,导致修正效果不佳。为此,本文提出一种广义联合偏差修正方法,相较于现有研究最常用的单变量QM修正方法以及固定多变量修正顺序的JBC修正方法,该方法充分考虑到流域尺度降水和气温双变量的时空相关性,并结合其对径流的主导作用对变量修正顺序进行动态调整,实现了QM法和JBC法的优势互补。在澜沧江-湄公河流域的应用表明：考虑降水、气温相关性可显著改善降水和温度极值的修正效果,尤其是5、6月份,修正后气象与实测数据的纳什系数提升了0.5以上;考虑气象要素的修正次序显著降低了修正后的降水和温度频率分布及均值偏差;利用修正后的气象数据驱动分布式水文模型时,部分月份的径流模拟精度提升了54.3%。 相似文献
12.
Based on the service specifications of the Open Geospatial Consortium (OGC), Spatial Data Infrastructures (SDI) support the visualization, access to vector and raster data or managing and search for spatial data. A standard for distributed spatial data processing was missing for a long time. This issue was addressed by the development of the OpenGIS Web Processing Service (WPS) specification. However, to process and analyze massive Digital Elevation Models (DEM) computing power and disk memory are scarce commodities. Here we show that Grid Computing in combination with OGC Web Services (OWS) is well suited to accomplish high processing performance and storage capacity for large-scale processing tasks of the geo community. To process these massive amounts of geo-data we develop terrain processing services which are made available as grid services. Our results will be demonstrating how to bridge the gap between the grid world and the OGC world for more sophisticated terrain processing. 相似文献
13.
黄土高原地区的土壤侵蚀具有水力侵蚀和重力侵蚀相伴发生的特点,在大多数针对黄土高原的侵蚀模型研究中未考虑重力侵蚀的影响,使得模型的模拟精度较差。针对黄土高原的侵蚀特点,采用量化影响重力侵蚀发生的主要因素,确定重力侵蚀发生的具体沟道栅格单元的方法,从而考虑了重力侵蚀对产输沙过程的影响。所建立的分布式土壤侵蚀模型以逐网格汇流的水文模型为基础,采用逐网格侵蚀输沙的模拟方式,能够模拟上方来水来沙对侵蚀输沙的影响。经小理河流域实测资料验证,模型具有一定的计算精度。 相似文献
14.
在充分研究了水电工程现有数据的基础上,根据水电工程数据的分布特点,提出了基于网格分块的三维地质体建模方法。并根据地表数据和地质数据的不同,分别提出了基于等高线切割的地表数据分块算法以及基于钻孔数据分布特点的地质数据分块算法,进一步利用边界缝合方法解决了分块引起的裂缝问题。在建立好三维地质体模型的基础上,根据网格分块的特点针对性的提出了基于网格分块的快速开挖算法。并分析了网格分块建立的地质模型比传统的地质模型的优势。通过西南某水电枢纽工程实例,检验了网格分块地质体建模的正确性和可行性。 相似文献
15.
《Atmósfera》2014,27(2):215-225
Using an air quality model, this study shows how emissions from the “Miguel Hidalgo” refinery of Petróleos Mexicanos (Pemex) and the thermoelectric plant “Francisco Pérez Ríos” of the Comisión Federal de Electricidad (CFE, Federal Electricity Commission) in Tula, Hidalgo influence the atmosphere of the Mexico City Metropolitan Area (MCMA). The model couples meteorology and chemistry. The weather scenario encompasses the period from October 20-28, 2005. Two scenarios are compared: the first assumes a 40% reduction in emissions of NOx, SO2, and volatile organic compounds (VOCs) from the Tula complex (reduction scenario), and the second considers the scenario without reduction (baseline scenario). The model is compared with measurements of the Red Automática de Monitoreo Atmosférico (Automatic Environmental Monitoring Network). We observe that under certain weather conditions, the energy sector of Tula, Hidalgo affects the air quality in the MCMA. The reduction scenario is effective in reducing SO2 concentrations; however, despite a 40% decrease in the emissions of ozone precursors, their concentrations in the MCMA did not decrease. 相似文献
16.
17.
Real-time storm detection and weather forecast activation through data mining and events processing 总被引：1，自引：0，他引：1
Xiang Li Beth Plale Nithya Vijayakumar Rahul Ramachandran Sara Graves Helen Conover 《Earth Science Informatics》2008,1(2):49-57
Each year across the USA, destructive weather events disrupt transportation and commerce, resulting in both loss of lives
and property. Mitigating the impacts of such severe events requires innovative new software tools and cyberinfrastructure
through which scientists can monitor data for specific severe weather events such as thunderstorms and launch focused modeling
computations for prediction and forecasts of these evolving weather events. Bringing about a paradigm shift in meteorology
research and education through advances in cyberinfrastructure is one of the key research objectives of the Linked Environments
for Atmospheric Discovery (LEAD) project, a large-scale, interdisciplinary NSF funded project spanning ten institutions. In
this paper we address the challenges of making cyberinfrastructure frameworks responsive to real-time conditions in the physical
environment driven by the use cases in mesoscale meteorology. The contribution of the research is two-fold: on the cyberinfrastructure
side, we propose a model for bridging between the physical environment and e-Science1 workflow systems, specifically through events processing systems, and provide a proof of concept implementation of that model
in the context of the LEAD cyberinfrastructure. On the algorithmic side, we propose efficient stream mining algorithms that
can be carried out on a continuous basis in real time over large volumes of observational data.
1 e-Science is used to describe computationally intensive science that is typically carried out in highly distributed network 相似文献
18.
19.
为提高网格环境下海量空间数据管理与并行化处理效率,将网格环境下的分布并行处理技术与空间索引相融合,提出了一种空间索引框架(grid slot and hash R tree,GSHR-Tree).该索引树结构基于散列hash表和动态空间槽,结合R树结构的范围查询优势和哈希表结构的高效单key查询,分析改进了索引结构的组织和存储.构造了适合于大规模空间数据的网格并行空间计算的索引结构,该索引树算法根据空间数据划分策略,动态分割空间槽,并将它们映射到多个节点机上.每个节点机再将其对应空间槽中的空间对象组织成R树,以大节点R树方式在多个节点上分布索引数据.以空间范围查询并行处理的系统响应时间为性能评估指标,通过模拟实验证明,该GSHR-Tree索引满足了当前网格环境空间索引的需要,并具有设计合理、性能高效的特点. 相似文献
20.
基于CMIP5和VIC模型的长江上游主要水文过程变化趋势预测 总被引：2，自引：0，他引：2
基于长江上游流域82个气象站点的实测数据和国际耦合模式比较计划第5阶段(Coupled Model Intercomparison Project phase 5,CMIP5)的2种排放情景下的8个气候模式1961～2099年的降水、气温数据,通过等距离累积分布函数法(Equidistance Cumulative Distribution Function Method,EDCDFm)进行气候模式的统计降尺度。在此基础上,构建0.5°×0.5°网格空间分辨率的可变下渗容量水文模型(Variable Infiltration Capacity,VIC),对历史流量进行模拟,并进一步模拟分析长江上游流域2006～2099年径流量、蒸散发的时空演变趋势。结果表明:VIC水文模型能够较好地模拟研究区的水文过程,从长江上游流域未来时期(2006～2099)主要水文过程变化趋势的预测来看,径流量变化趋势不明显、蒸散发呈增加趋势。此研究对于合理规划配置长江流域水资源及为气候影响评价和决策系统提供科技支撑具有重要意义。 相似文献