Add tutorial on selecting qubits and calibrating for an experiment #4108

rmlarose · 2021-05-14T05:19:15Z

This tutorial shows how to run XEB and Floquet calibrations and compare the results with a Loschmidt echo.

Still a bit WIP but asking @mrwojtek @mpharrigan and @XiaoMiQC for initial comments.

Example data saved with disclaimer - ok'd by @karlunho.

review-notebook-app · 2021-05-14T05:19:18Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

mrwojtek · 2021-05-14T13:16:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


@mpharrigan, should the defaults be set to the cycle depths above in Cirq?

Why do you use just one processor?

Reply via ReviewNB

Not specifying n_processes=1 leads to a hanging response on Colab.

You can probably see improved performance if you set characterize_{angle} to false except for zeta, chi, gamma (like is done for floquet). Although floquet is also set to false fo chi

Another option is to change the ndefault in XEB wrapper to n_processes=max(1, #of processors - 1)

it's currently the multiprocessing default of # of processors

mrwojtek · 2021-05-14T13:16:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


I would be against doing characterizations together: they take a long time and the fresher the results are, the better resulting quality.

After each characterization the corresponding calibrated circuits should be executed immediately. The code here should be merged with the code below with loop over options being the outermost one.

Reply via ReviewNB

Good feedback, thanks! Perhaps this will make the results closer to what is expected...

I would split these up so folks following along can pick one or the other to copy-paste without having to disentangle a for loop

Agree. It's split up now and should be easy to skip/comment out either calibration.

I still do both calibrations then run Loschmidt instead of [calibrate, Loschmidt] x 2. Results are looking much better now I think, so maybe this is fine. Let me know what you think @mrwojtek.

I see what you've been trying to do. I'll do some adjustments to the calibration methods to make your workflow easier.

However, I propose some refactor which will simplify things significantly in this tutorial. It will also probably improve the performance (I expect that Floquet calibration can be improved if ran separately):

"Floquet calibration" section should just have the first cell, the one which defines floquet_options. Remove the second cell.
"XEB calibration" section should just have the first cell, the one which defines xeboptions. Remove the second cell.
In the "Run Loschmidt echo benchmark" you can just define:
calibrations = {
  "No calibration": None,
  "Floquet calibration": floquet_options,
  "XEB calibration": xeb_options`
}

Then you just need to do:
batch_to_compensate, characterization_requests = cg.prepare_characterization_for_circuits_moments(loschmidt_echo_batch, options)

characterizations = cg.run_calibrations(
  characterization_requests,
  sampler,
)

batch = [
      cg.make_zeta_chi_gamma_compensation_for_moments(circ, characterizations).circuit
      for circ in batch_to_calibrate
    ]

results = sampler.run_batch(programs=batch, repetitions=nreps)

This is now done, with a note to update after https://github.com//pull/4118 is merged.

I also added another calibration for the user-defined circuit, instead of reusing one of the characterizations from the Loschmidt echo, as we discussed offline.

@rmlarose I don't see the changes mentioned here in this document? Did you forget to push your changes?

Just to document it here, they were in dfbe326 which was reverted.

mrwojtek · 2021-05-14T13:16:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


I think that it might be better to use sampler.run_batch, there is less boiler-plate that way. Is there a particular reason that you're using engine.run_batch?

Reply via ReviewNB

Agree. I tried moving to sampler.run_batch but, if I recall correctly, the return type was different and didn't work with to_ground_state_prob. Just being lazy though, I'll move to sampler only semantics.

All engines replaced by samplers.

mrwojtek · 2021-05-14T13:16:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


This is strange, the results are unexpected. Does your branch include https://github.com//pull/4092 which contain a fix for poor XEB performance?

Reply via ReviewNB

That was in the v0.11 release, so yes it should. (My branch is just cirq==0.11.0.)

mpharrigan · 2021-05-14T23:03:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


"assuming Application Default Credentials are setup." -> set up

Reply via ReviewNB

Done (note this was taken from https://quantumai.google/cirq/tutorials/google/start).

Yeah I assumed this was a copy paste but good to have it fixed here at least

mpharrigan · 2021-05-14T23:03:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


worth opening an issue to support this function natively? namely: provide a list of metrics and generate a grid of plots. Note that if you add a third metric it looks rather squished.

Reply via ReviewNB

At least for tutorials I think this is much better than plotting them sequentially and having to scroll through everything. Not sure if useful outside of this, though.

mpharrigan · 2021-05-14T23:03:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


A rather important bug, that I think is prevalent in the loschmidt echos notebook as well: try doing cycles=1. How many two-qubit layers do you see? How about cylces=2?

This is because you are using grid_staggered_pattern but a horizontal line of qubits (instead of a grid). Related but not necessarily sufficient: https://github.com//issues/3833

Options:
have the random generation function ignore the gridinteractionlayers that don't produce interactions. This would solve the issue at hand but wouldn't produce optimal circuits if you had a line of qubits that was snaked on a diagonal or something
Have a special function or pattern to do random circuits on linear topologies
Have a be-all-end-all random circuit generation function that works on arbitrary planar topologies by doing edge coloring to figure out activation orders.
Make cycles always a multiple of 4

Reply via ReviewNB

Thanks for pointing this out. Discovering this was the main reason the Loschmidt tutorial took so long.

For now I kept cycles as a multiple of 4, but the topology is also no longer linear.

Would you really call this a bug, though? It makes computing the per-two-qubit-gate error harder, but every circuit in the Loschmidt batch has the same number of two-qubit gates, so its valid to compare.

It's surprising. I added a variant of the above comment to #3342

mpharrigan · 2021-05-14T23:03:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


yeah definitely use sampler. Also you should pass project_id to get_engine (or get_engine_sampler) since you go through the effort of allowing it to be input as a variable.

I wish we had some indication of progress or visual indication that something is happening when you run this cell.

Reply via ReviewNB

mpharrigan · 2021-05-14T23:03:41Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


'o-' is a little heavy-handed for so many lines. Maybe '.-' or even '-'. Then you can back off the lw for average. 5 is a lot

Reply via ReviewNB

mpharrigan · 2021-05-14T23:03:42Z

docs/tutorials/google/advanced_calibration.ipynb

@@ -0,0 +1,799 @@
+{


use cycle= to document what 10 is

Reply via ReviewNB

rmlarose · 2021-05-18T04:13:48Z

Other changes:

Remove initial Loschmidt echo and do the uncalibrated one at the end. Suggested by Alan as the initial Loschmidt was not part of the workflow apparently.
Use 10 qubits instead of 4. Improvements from calibration are much clearer and more consistent now.

This reverts commit dfbe326.

mpharrigan · 2021-05-20T00:44:58Z

docs/_book.yaml

@@ -186,6 +186,8 @@ upper_tabs:
        path: /cirq/tutorials/google/echoes
      - title: "Visualizing Calibration Metrics"
        path: /cirq/tutorials/google/visualizing_calibration_metrics
+      - title: "Advanced calibration"


nit: this doesn't really match the title given in the notebook

updated to match notebook title

@mrwojtek

…uantumlib#4108) This tutorial shows how to run XEB and Floquet calibrations and compare the results with a Loschmidt echo. Still a bit WIP but asking @mrwojtek @mpharrigan and @XiaoMiQC for initial comments. Example data saved with disclaimer - ok'd by @karlunho.

@mrwojtek

…uantumlib#4108) This tutorial shows how to run XEB and Floquet calibrations and compare the results with a Loschmidt echo. Still a bit WIP but asking @mrwojtek @mpharrigan and @XiaoMiQC for initial comments. Example data saved with disclaimer - ok'd by @karlunho.

Add tutorial.

c2b24e3

rmlarose requested review from cduck, vtomole, wcourtney and a team as code owners May 14, 2021 05:19

rmlarose requested a review from maffoo May 14, 2021 05:19

google-cla bot added the cla: yes Makes googlebot stop complaining. label May 14, 2021

mrwojtek reviewed May 14, 2021

View reviewed changes

mpharrigan reviewed May 14, 2021

View reviewed changes

rmlarose mentioned this pull request May 18, 2021

Default behavior for missing calibration data #4117

Closed

rmlarose added 2 commits May 17, 2021 23:51

Updates + review comments.

9bcee98

Review comments.

c0c2576

More review comments.

dfbe326

rmlarose requested review from mpharrigan and mrwojtek May 19, 2021 16:12

rmlarose added 4 commits May 19, 2021 14:24

Revert "More review comments."

b291cf0

This reverts commit dfbe326.

Add note about needing all two-qubit pairs in the example circuit.

b62e756

Rename function.

a7e90c2

Explain what best results means.

4b3c2b8

mpharrigan approved these changes May 20, 2021

View reviewed changes

Update TOC.

0dfd0d1

mpharrigan added the automerge Tells CirqBot to sync and merge this PR. (If it's running.) label May 20, 2021

CirqBot added the front_of_queue_automerge CirqBot uses this label to indicate (and remember) what's being merged next. label May 20, 2021

Merge branch 'master' into add-qcs-tutorial

3ee4ee8

CirqBot merged commit 51d6884 into quantumlib:master May 20, 2021

CirqBot removed automerge Tells CirqBot to sync and merge this PR. (If it's running.) front_of_queue_automerge CirqBot uses this label to indicate (and remember) what's being merged next. labels May 20, 2021

Add tutorial on selecting qubits and calibrating for an experiment #4108

Add tutorial on selecting qubits and calibrating for an experiment #4108

Uh oh!

Conversation

rmlarose commented May 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

review-notebook-app bot commented May 14, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrwojtek May 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rmlarose commented May 18, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rmlarose commented May 14, 2021 •

edited

Loading

mrwojtek May 14, 2021 •

edited

Loading