283 views

Skip to first unread message

May 31, 2021, 8:25:32 PMMay 31

to OpenQuake Users

Hello All,

What is the right way to get a mean disaggregation?
is mean disaggregation calculated by default? i want to calculate the true mean disaggregation so please suggest the proper way.

regards

Sujan

Jun 1, 2021, 12:34:14 PMJun 1

to OpenQuake Users

will this code give me true mean disaggregation or do we need to list all of the realizations within *rlz_index* or set *num_rlzs_disagg* to the total number of branches in order to obtain the true mean deaggregation. your help and suggestion will be highly appreciated.

I am using this code...

[General]

description = Disaggregation450

calculation_mode = disaggregation

random_seed = 23

max_sites_disagg=83

[Geometry]

sites_csv = MV_sites.csv

[erf]

rupture_mesh_spacing = 5.0

width_of_mfd_bin = 0.1

area_source_discretization = 10.0

[site_params]

reference_vs30_type = measured

reference_vs30_value = 450.0

reference_depth_to_2pt5km_per_sec = 2.0

reference_depth_to_1pt0km_per_sec = 300.0

[Logic_tree]

source_model_logic_tree_file = ssms/simplifiedCollapsed/logicTree/source_model_logic_tree.xml

gsim_logic_tree_file = gmms/logicTree/Vs30_450.xml

number_of_logic_tree_samples = 0

[Hazard_calculation]

intensity_measure_types_and_levels = {"PGA": logscale(0.001,6.0, 40),

"SA(0.2)": logscale(0.001,6.0, 40),

"SA(2.0)": logscale(0.001,4.0, 40),

truncation_level = 5

investigation_time = 50.0

#maximum_distance = 500

[disaggregation]

poes_disagg = 0.02, 0.05, 0.1, 0.2, 0.4, 0.5

mag_bin_width = 1

distance_bin_width = 20.0

coordinate_bin_width = 2.5

num_epsilon_bins = 5

disagg_outputs = Mag, Dist, Mag_Dist, Mag_Dist_Eps

disagg_by_src = false

[output]

individual_curves = true

mean = true

quantiles = 0.1 0.5 0.9

hazard_maps = true

uniform_hazard_spectra = true

poes_disagg = 0.02, 0.05, 0.1, 0.2, 0.4, 0.5

export_dir = /tmp

Jun 3, 2021, 1:31:42 AMJun 3

to OpenQuake Users

You need to specify rlz_index or num_rlzs_disagg.

Michele

Jun 3, 2021, 5:48:16 AMJun 3

to OpenQuake Users

Thanks, Michele, but Will you please tell me the way to do that. what I have to write and where.

Jun 5, 2021, 6:55:35 AMJun 5

to OpenQuake Users

still waiting

Jun 10, 2021, 7:15:21 PMJun 10

to OpenQuake Users

Hi Michele

Does this
number_of_logic_tree_samples = 0 solve the problem . or what s the way to
specify rlz_index

regards

Sujan

On Thursday, June 3, 2021 at 1:31:42 AM UTC-4 Michele Simionato wrote:

Jun 11, 2021, 11:58:55 PMJun 11

to OpenQuake Users

It is documented in the manual and there is also an example in the code base.

Jun 27, 2021, 11:11:18 AMJun 27

to OpenQuake Users

I tried as of manual by considering num_rzls_disagg

but the result I am getting is the same as the previous one when I did not use
num_rzls_disagg

so is it true mean disaggregation?

Sujan

Jul 3, 2021, 11:25:19 PMJul 3

to OpenQuake Users

I'm having trouble following this thread. The orginal poster Sujan, wants to calculate the true mean disaggregation, it is not clear why setting the realisation number obtains this?

Further, it is not clear how the weightings of different GMPEs are considered when performing the disaggregation. Bazzurro & Cornell (1999) provide the following: *“When
epistemic uncertainty is included in the PSHA calculations, in principle, one
could disaggregate the hazard from each considered seismicity model. In
realistic applications, this is highly impractical given the very large number
of cases typically considered. However, the same procedure outlined earlier can
still be applied to disaggregate, for example, the mean hazard.
Computationally, this can be done by multiplying the hazard contribution in
each bin by the weight assigned to the model under consideration. Because the
sum of all the weights adds up to 1, the results of this operation provide the
relative contributions to the mean hazard.”*

Does OpenQuake do this?

Does OpenQuake do this?

Paper link: 010435015.TIF (utl.pt)

My second question relates to calculation of epsilon, Harmsen (BSSA 91(6), 2001) notes how the conditional weighted value of epsilon is calculated. Does OpenQuake do this when a 'non trivial' GMPE logic tree is present?

Paper Link: C10. Harsmen(2001).pdf (memphis.edu)

Finally, Lin & Baker (2010) describe the means to conduct disaggregation based on GMPE, in order to calculate the conditional mean spectrum from a hazard analysis that uses multiple GMPEs.

Is this possible with OpenQuake, and if not could it be implemented in a future update? It would be very valuable for calculating the conditional mean spectrum, and conditional intensity measures based on a PSHA with a 'non trivial' GMPE logic tree (which is best practice).

Paper Link: Probabilistic seismic hazard deaggregation of ground motion prediction models (stanford.edu)

Thanks and Regards,

Merrick

Jul 5, 2021, 6:44:52 AMJul 5

to OpenQuake Users

Dear Merrick,

For question 1 you might want to read the OpenQuake underlying hazard science file (https://www.globalquakemodel.org/gempublications/OpenQuake-Underlying-Hazard-Science) section 2, and in particular 2.4.2. In essence, Openquake does NOT provide the classical results such as Bazzurro & Cornell (1999) BUT they can be converted manually using the procedure described in that section 2.4.2. When I performed my disaggregation analyses I used the version 3.4.0 (the one available at that time) that provived the disaggregation for all the realizations. Then, I manually calculated the mean disaggregation based on those realizations. I am unsure of how the latest versions work, but I can tell you that OQ currently does not provide the mean disaggregation and it does not calculate the conditional mean spectrum. You need to caculate manually the M,R,epsilon that most contribute to the hazard (i.e. the modal values or the mean values, see Bazzurro & Cornell 1999) and then perform the calculations necessary for the CMS using the GMPE you need. However, I do agree that an update regarding the computation of the CMS would be great.

I am unsure about question 2, you either need the developer to answer that for you or search in the .py files about the disaggregation. The great think about OQ is that it is REALLY opensource, such that you can see every equation implemented.

Best,

Gianluca

Jul 5, 2021, 4:42:55 PMJul 5

to OpenQuake Users

Gianluca,

Thanks for your response and confirmation of a few details, and suggestion to manually calculate the mean disaggregation from all the realisations.

Unfortunately the link you provide gives an error when I try to download the document (unsecure connection warning, and if persist, a "Disalllowed Host" message)

"DisallowedHost at /public/wix-new-website/pdf-collections-wix/publications/OQ Hazard Science 1.0.pdfInvalid HTTP_HOST header: 'storage-scw1.globalquakemodel.org'. You may need to add 'storage-scw1.globalquakemodel.org' to ALLOWED_HOSTS." etc.

"DisallowedHost at /public/wix-new-website/pdf-collections-wix/publications/OQ Hazard Science 1.0.pdfInvalid HTTP_HOST header: 'storage-scw1.globalquakemodel.org'. You may need to add 'storage-scw1.globalquakemodel.org' to ALLOWED_HOSTS." etc.

I would be very grateful if you could share this document with me via email. My address is my full name (as one word, lowercase, at gmail dot com).

Regards,

Merrick

Jul 6, 2021, 12:02:58 AMJul 6

to OpenQuake Users

Hi, it is true, now the download link for the hazard science document stopped working, could someone from GEM team correct it?

I have an older copy but cannot attach due to size limit in this forum.

Peter

Jul 6, 2021, 4:13:36 AMJul 6

to OpenQuake Users, Jephraim Oro

Merrick,

The problem is now fixed. You should be able to download the documentation at the following link

https://www.globalquakemodel.org/oq-get-started

Marco

--

You received this message because you are subscribed to the Google Groups "OpenQuake Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openquake-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openquake-users/09ad982c-8eb8-44ba-b02d-adac8e6a0b03n%40googlegroups.com.

Jul 6, 2021, 7:13:12 AMJul 6

to OpenQuake Users

thanks for reopening this thread,

it's true that OQ disaggregation calculator does not provide the result same as the traditional disaggregation calculator. I searched the OpenQuake Users groups a found the post response by Damiano Monelli regarding this issue with his Magnitude example. I did the same for my M_R to get the traditional result and I attached the excel sheet with the formula here

but i am still confused /suffering to calculate the true mean disaggregation.. i did all the things written in the manual see post in the upper section. i am still looking for help on this one. if someone knows the procedure please suggest me.

Jul 6, 2021, 5:59:11 PMJul 6

to OpenQuake Users

Thanks Marco for fixing the link, I have successfully downloaded the document.

I realise now from reading your responses, that my question above has confused two aspects related to the disaggregation.

1) The way OpenQuake provides disaggregation output. Yes I have also read Damiano Monelli's post to calculate the traditional % contribution. The *mean magnitude* and distance (or mode) can be determined from this output, per B&C99. As Gianluca advises, this conversion is described in mathematical notation in the hazard science document.

2) However, my current issue I am trying to understand/ resolve is that when a GMPE logic tree is adopted, OpenQuake does not calculate the disaggregation from the *mean (or some fractal such as the median) hazard curve*, but the hazard curve of the realisation closest to the mean. For a non-trivial logic tree (e.g. adopting 3x GMPE for each TRT: shallow crustal, s. slab, s. int) it will not be clear which realisation (i.e. which branch of the logic tree) is selected as being 'closest to the mean', for performing the disaggregation, or how the disaggregation should be calculated (i.e. different GMPE will result in different % contributions, and different epsilon values for the same mean hazard curve - they need to be weighted appropriately to obtain the mean).

I presume that one could extract disaggregation data from each realisation (potentially many results for complex logic trees!) and manually calculate weighted contributions for each M, R, epsilon bin across all realisations. It's a bit laborious to do this, and it would be good if there were an option for the OpenQuake engine to provide the option to deliver this output - perhaps a wish list for now. I need to think and learn how I can combine the available disaggregation outputs, e.g. (M, Lat, Lon) & (TRT, Lat, Lon)
across all realisations to obtain sufficient information to calculate the required M, R, epsilon statistics where a complex GMPE logic tree is employed.

Regards,

Merrick

Jul 7, 2021, 4:43:27 AMJul 7

to OpenQuake Users

asu...@gmail.com, your excel file seems good to me. For the "mean" disaggregation values, you might want to check this paper:

https://www.sciencedirect.com/science/article/pii/S0267726117300386which provides lots of details on the OQ disaggregation.

Merrick,

when I performed my disaggregation analyses, using version 3.4.0, it automatically gave the results for all the realizations (though it was not that efficient). I did a quick test on the latest version (way more efficient speed-wise), and if you want to do the same for all realizations you need to add the command Michele said in the job.ini file, which is in the OQ manual (pag. 65 of version 3.11.2):

rlz_index = 0,1,2,3,4...,N

where N is the number of your realizations. If N is large, you might want to use the default realization, i.e., the one closest to the mean that OQ currently provides.

It is indeed laborious, thus I suggest to build a post-processing code that performs all this calculation automatically, based on the .csv files.

About this question: ". For a non-trivial logic tree (e.g. adopting 3x GMPE for each TRT: shallow crustal, s. slab, s. int) it will not be clear which realisation (i.e. which branch of the logic tree) is selected as being 'closest to the mean'". Actually, when you download the output results using ONLY the realization closest to the mean (i.e., without the rlz_index command), the files are named like this:

rlz_xx_PGA-sid-0-poe-yy_Dist_zz

where xx is the index of the realization closest to the mean, yy is the poe you choose and zz the calculation number (of course this is similar for spectral accelerations and other disaggregations type). However, in the header of the csv file, you can also find the logic tree path (gsimlt_path='b3 b32 b22' in my case). In addition, another output is the "realizations" csv, where other information can be found. In general, in the Full report file (the rst file) you can find all the information you need (e.g. for the GSIM xx, the corresponding realizations are n1,n2,n3 etc).

Jul 7, 2021, 5:31:16 AMJul 7

to OpenQuake Users

Gianluca made a great post, thanks!

I will only add than instead than listing all the realizations manually with rlz_index, you can just specify num_rlzs_disagg = R where R is the total number of realizations of your calculation and then you will get all the possible outputs.

There is also an extract API to get the mean disaggregation considering all the realizations, but it is still undocumented.

Michele

Jul 7, 2021, 5:49:21 AMJul 7

to OpenQuake Users

The big issue with the disaggregation outputs is that they are too many. An option could be to export all of them in a single file in HDF5 format. Another is to reduce the number of CSV files by following the idea in https://github.com/gem/oq-engine/issues/6753. Here we are lacking user input: how do you want the realization outputs to be exported? Suggestions are appreciated!

Michele

Jul 7, 2021, 6:35:03 AMJul 7

to OpenQuake Users

I can see the problem with handling all those files! I was lucky enough to have only 54 realizations (thus 432=54*8 csv files) for 4 POE, which I later post-processed. The way I handled the files, though, was a bit lame, meaning that I performed the analyses one POE at a time, such that I had the results in separate folders. I am unfamiliar with the HDF5 format, but for the csv format I agree that maybe some results could be joined together, but not necessarily per column. For instance, since the POE are usually limited (maybe corresponding to return periods of 2500, 1460, 75, 50 years and so on) you could separate those results per sheet. If only one POE is used, then the csv would have one sheet, and so on. This way, considering the example you illustrated you would have 70000/5 = 14000 files, which is still a huge number, but more manageable. If the number of sites and disaggregation types needed by the user are also low, then this sheet methodology could be also used, but it would fail otherwise, as you would have a csv file with N*P*t sheets (where t is the number of disaggregation types you need). On the other hand, if you have M>P, so more IMTs than POEs, you would have 70000/10 = 7000 files each one with 10 sheets, for M=10.

I do not know if it is possible and how intuitive might be, but maybe the user could have the option to decide how to join the results. In my case, it would have been great to have the results of different POEs in different sheets. Another user maybe would find it more interesting to join them for site (e.g. they only have 1 POE).

I think that a couple of csv files containing the information of ALL the results might be confusing, unless the columns are really well organized. In addition, there is the parsing problem you mentioned.

However, if the mean disaggregation is done internally, there would be no need at all to export all the realizations, unless for verification problems. As it is now, I think is a good compromise, as you divide the total number by R.

Gianluca

Jul 28, 2021, 5:16:42 PMJul 28

to OpenQuake Users

Hello, thanks all for your replies, it has helped my understanding of this issue.

For post-processing to determine conditional ground motion intensity parameters (e.g. for conditional spectrum, or conditional CAV, AI, PGV etc), would require Mag_Dist_Eps_TRT, or at least Mag_Dist_TRT (the latter allowing epsilon to be calculated manually for each M-R bin using GMPEs corresponding to the different TRTs). Can this be added to a wish list for the next update to the disaggregation calculator? (it is admittedly beyond my humble coding capability)

For post-processing to determine conditional ground motion intensity parameters (e.g. for conditional spectrum, or conditional CAV, AI, PGV etc), would require Mag_Dist_Eps_TRT, or at least Mag_Dist_TRT (the latter allowing epsilon to be calculated manually for each M-R bin using GMPEs corresponding to the different TRTs). Can this be added to a wish list for the next update to the disaggregation calculator? (it is admittedly beyond my humble coding capability)

Regards,

Merrick

Aug 23, 2021, 3:05:49 PMAug 23

to OpenQuake Users

I second Merrick's request. I need to do deaggregation by source and there really should be a Mag_Dist_Eps_TRT option. Please let me know if there is some other way of getting this.

Aug 29, 2021, 5:26:53 PMAug 29

to OpenQuake Users

Daniel,

The closest I have been able to 'fudge' is to use Lon-Lat-Mag, and Lon-Lat-TRT with a closely spaced Lon-Lat Binning (as close as the deisaggregation calculator will allow, and you may need to limit the maximum distance to get the precision required - in my case I needed to reduce to 200km max distance, even though the calculator happily gave M-R disaggregation to 500km distance) to allow you to manually approximate the M-R disaggregation from Lon-Lat-TRT (through calculating R from the site Lon-Lat and the source Lon-Lat using the Haversine formula). Then do the same for Lon-Lat-TRT with the same distance binning. By comparing the two disaggregation plots you can estimate the % contribution for a particular distance that is from shallow crustal vs subduction slab vs interface. By comparing the two plots, you wont know the exact breakdown by magnitude, and you will have to make some broad assumptions and apply weights based on the contributions reported in the plots e.g. to calculate epsilon manually for each M-R bin, and undertake intensity measure correlations, but it is as close as one can get for now with OpenQuake I believe.

The closest I have been able to 'fudge' is to use Lon-Lat-Mag, and Lon-Lat-TRT with a closely spaced Lon-Lat Binning (as close as the deisaggregation calculator will allow, and you may need to limit the maximum distance to get the precision required - in my case I needed to reduce to 200km max distance, even though the calculator happily gave M-R disaggregation to 500km distance) to allow you to manually approximate the M-R disaggregation from Lon-Lat-TRT (through calculating R from the site Lon-Lat and the source Lon-Lat using the Haversine formula). Then do the same for Lon-Lat-TRT with the same distance binning. By comparing the two disaggregation plots you can estimate the % contribution for a particular distance that is from shallow crustal vs subduction slab vs interface. By comparing the two plots, you wont know the exact breakdown by magnitude, and you will have to make some broad assumptions and apply weights based on the contributions reported in the plots e.g. to calculate epsilon manually for each M-R bin, and undertake intensity measure correlations, but it is as close as one can get for now with OpenQuake I believe.

Regards,

Merrick

Sep 21, 2021, 9:58:53 AMSep 21

to OpenQuake Users

On Tuesday, July 6, 2021 at 11:59:11 PM UTC+2 Merrick Taylor wrote:

Thanks Marco for fixing the link, I have successfully downloaded the document.I realise now from reading your responses, that my question above has confused two aspects related to the disaggregation.1) The way OpenQuake provides disaggregation output. Yes I have also read Damiano Monelli's post to calculate the traditional % contribution. Themean magnitudeand distance (or mode) can be determined from this output, per B&C99. As Gianluca advises, this conversion is described in mathematical notation in the hazard science document.2) However, my current issue I am trying to understand/ resolve is that when a GMPE logic tree is adopted, OpenQuake does not calculate the disaggregation from themean (or some fractal such as the median) hazard curve, but the hazard curve of the realisation closest to the mean. For a non-trivial logic tree (e.g. adopting 3x GMPE for each TRT: shallow crustal, s. slab, s. int) it will not be clear which realisation (i.e. which branch of the logic tree) is selected as being 'closest to the mean', for performing the disaggregation, or how the disaggregation should be calculated (i.e. different GMPE will result in different % contributions, and different epsilon values for the same mean hazard curve - they need to be weighted appropriately to obtain the mean).I presume that one could extract disaggregation data from each realisation (potentially many results for complex logic trees!) and manually calculate weighted contributions for each M, R, epsilon bin across all realisations. It's a bit laborious to do this, and it would be good if there were an option for the OpenQuake engine to provide the option to deliver this output - perhaps a wish list for now. I need to think and learn how I can combine the available disaggregation outputs, e.g. (M, Lat, Lon) & (TRT, Lat, Lon) across all realisations to obtain sufficient information to calculate the required M, R, epsilon statistics where a complex GMPE logic tree is employed.

I am going back to this old email since there has been some been some work on the disaggregation recently.

1. since many releases ago you can specify the realizations you are interested in with the variable rlz_index (example: https://github.com/gem/oq-engine/tree/master/openquake/qa_tests_data/disagg/case_2)

2. in version 3.12 exporting the disaggregation outputs produces a CSV file for each realization specified in rlz_index, but in current master all realizations are exported in the same file, so it is easier to compute the means

(example: https://github.com/gem/oq-engine/blob/master/openquake/qa_tests_data/disagg/case_2/expected_output/Mag-0.csv, where there are 4 realizations and all of them are listed in the rlz_index)

3. in current master we also have implemented a conditional spectrum calculator (example: https://github.com/gem/oq-engine/tree/master/openquake/qa_tests_data/conditional_spectrum/case_1)

The new features are not yet documented, but they will before the release of engine 3.13.

HTH,

Michele

Sep 21, 2021, 3:02:31 PMSep 21

to OpenQuake Users

Thanks, Michele

Reply all

Reply to author

Forward

0 new messages