FIT Profiler Report

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Action                                                                                                                                                                                                     	|  Mean duration (s)	|  Num calls      	|  Total time (s) 	|  Percentage %   	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  Total                                                                                                                                                                                                      	|  -              	|  884625         	|  3115.8         	|  100 %          	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|  run_training_epoch                                                                                                                                                                                         	|  148.61         	|  20             	|  2972.2         	|  95.39          	|
|  run_training_batch                                                                                                                                                                                         	|  0.12755        	|  20000          	|  2551.1         	|  81.875         	|
|  [LightningModule]NeRFSystem.optimizer_step                                                                                                                                                                 	|  0.12712        	|  20000          	|  2542.3         	|  81.594         	|
|  [Strategy]SingleDeviceStrategy.training_step                                                                                                                                                               	|  0.031176       	|  20000          	|  623.51         	|  20.011         	|
|  [Strategy]SingleDeviceStrategy.validation_step                                                                                                                                                             	|  0.92861        	|  260            	|  241.44         	|  7.7487         	|
|  [TrainingEpochLoop].train_dataloader_next                                                                                                                                                                  	|  0.0073439      	|  20000          	|  146.88         	|  4.7139         	|
|  [LightningModule]NeRFSystem.setup                                                                                                                                                                          	|  131.71         	|  1              	|  131.71         	|  4.2273         	|
|  [Strategy]SingleDeviceStrategy.backward                                                                                                                                                                    	|  0.005273       	|  20000          	|  105.46         	|  3.3846         	|
|  [Callback]TQDMProgressBar.on_train_batch_end                                                                                                                                                               	|  0.00053806     	|  20000          	|  10.761         	|  0.34537        	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_end                 	|  0.18694        	|  20             	|  3.7387         	|  0.11999        	|
|  [Strategy]SingleDeviceStrategy.batch_to_device                                                                                                                                                             	|  0.0001631      	|  20260          	|  3.3043         	|  0.10605        	|
|  [LightningModule]NeRFSystem.transfer_batch_to_device                                                                                                                                                       	|  9.9472e-05     	|  20260          	|  2.0153         	|  0.064679       	|
|  [EvaluationEpochLoop].val_dataloader_idx_0_next                                                                                                                                                            	|  0.00454        	|  260            	|  1.1804         	|  0.037884       	|
|  [LightningModule]NeRFSystem.optimizer_zero_grad                                                                                                                                                            	|  3.0648e-05     	|  20000          	|  0.61296        	|  0.019672       	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_end                 	|  2.0385e-05     	|  20000          	|  0.40769        	|  0.013085       	|
|  [LightningModule]NeRFSystem.configure_gradient_clipping                                                                                                                                                    	|  1.9822e-05     	|  20000          	|  0.39645        	|  0.012724       	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_end                                                                                                                                                 	|  7.1346e-06     	|  20000          	|  0.14269        	|  0.0045796      	|
|  [Callback]TQDMProgressBar.on_after_backward                                                                                                                                                                	|  6.183e-06      	|  20000          	|  0.12366        	|  0.0039688      	|
|  [Callback]TQDMProgressBar.on_before_optimizer_step                                                                                                                                                         	|  5.8778e-06     	|  20000          	|  0.11756        	|  0.0037729      	|
|  [Callback]TQDMProgressBar.on_batch_start                                                                                                                                                                   	|  5.6706e-06     	|  20000          	|  0.11341        	|  0.0036399      	|
|  [LightningModule]NeRFSystem.on_validation_start                                                                                                                                                            	|  0.055241       	|  2              	|  0.11048        	|  0.0035459      	|
|  [Callback]TQDMProgressBar.on_validation_batch_end                                                                                                                                                          	|  0.00041116     	|  260            	|  0.1069         	|  0.003431       	|
|  [Callback]TQDMProgressBar.on_before_zero_grad                                                                                                                                                              	|  4.8051e-06     	|  20000          	|  0.096102       	|  0.0030843      	|
|  [Callback]TQDMProgressBar.on_train_batch_start                                                                                                                                                             	|  4.4778e-06     	|  20000          	|  0.089556       	|  0.0028742      	|
|  [Callback]TQDMProgressBar.on_before_backward                                                                                                                                                               	|  4.4336e-06     	|  20000          	|  0.088672       	|  0.0028459      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_start                     	|  4.307e-06      	|  20000          	|  0.086139       	|  0.0027646      	|
|  [Callback]GradientAccumulationScheduler.on_batch_start                                                                                                                                                     	|  4.2228e-06     	|  20000          	|  0.084456       	|  0.0027105      	|
|  [Callback]GradientAccumulationScheduler.on_before_optimizer_step                                                                                                                                           	|  4.2127e-06     	|  20000          	|  0.084254       	|  0.0027041      	|
|  [Callback]GradientAccumulationScheduler.on_train_batch_start                                                                                                                                               	|  4.0635e-06     	|  20000          	|  0.08127        	|  0.0026083      	|
|  [Callback]GradientAccumulationScheduler.on_after_backward                                                                                                                                                  	|  4.0591e-06     	|  20000          	|  0.081182       	|  0.0026055      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_optimizer_step           	|  4.0192e-06     	|  20000          	|  0.080384       	|  0.0025799      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_batch_start               	|  4.0171e-06     	|  20000          	|  0.080343       	|  0.0025785      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_after_backward                  	|  4.0026e-06     	|  20000          	|  0.080051       	|  0.0025692      	|
|  [Callback]TQDMProgressBar.on_batch_end                                                                                                                                                                     	|  3.9792e-06     	|  20000          	|  0.079585       	|  0.0025542      	|
|  [Callback]GradientAccumulationScheduler.on_before_backward                                                                                                                                                 	|  3.9554e-06     	|  20000          	|  0.079108       	|  0.0025389      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_batch_end                       	|  3.9451e-06     	|  20000          	|  0.078902       	|  0.0025323      	|
|  [Callback]GradientAccumulationScheduler.on_batch_end                                                                                                                                                       	|  3.9336e-06     	|  20000          	|  0.078673       	|  0.0025249      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_backward                 	|  3.9143e-06     	|  20000          	|  0.078287       	|  0.0025126      	|
|  [Callback]GradientAccumulationScheduler.on_before_zero_grad                                                                                                                                                	|  3.9077e-06     	|  20000          	|  0.078154       	|  0.0025083      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_zero_grad                	|  3.8868e-06     	|  20000          	|  0.077736       	|  0.0024949      	|
|  [LightningModule]NeRFSystem.on_before_batch_transfer                                                                                                                                                       	|  3.1626e-06     	|  20260          	|  0.064073       	|  0.0020564      	|
|  [LightningModule]NeRFSystem.training_step_end                                                                                                                                                              	|  2.6215e-06     	|  20000          	|  0.05243        	|  0.0016827      	|
|  [LightningModule]NeRFSystem.on_train_batch_end                                                                                                                                                             	|  2.4617e-06     	|  20000          	|  0.049234       	|  0.0015801      	|
|  [Strategy]SingleDeviceStrategy.training_step_end                                                                                                                                                           	|  2.46e-06       	|  20000          	|  0.049201       	|  0.0015791      	|
|  [LightningModule]NeRFSystem.on_train_batch_start                                                                                                                                                           	|  2.4314e-06     	|  20000          	|  0.048629       	|  0.0015607      	|
|  [LightningModule]NeRFSystem.on_after_batch_transfer                                                                                                                                                        	|  2.395e-06      	|  20260          	|  0.048523       	|  0.0015573      	|
|  [LightningModule]NeRFSystem.on_after_backward                                                                                                                                                              	|  2.2617e-06     	|  20000          	|  0.045234       	|  0.0014517      	|
|  [Strategy]SingleDeviceStrategy.on_train_batch_start                                                                                                                                                        	|  2.2565e-06     	|  20000          	|  0.04513        	|  0.0014484      	|
|  [LightningModule]NeRFSystem.on_before_zero_grad                                                                                                                                                            	|  2.2128e-06     	|  20000          	|  0.044257       	|  0.0014204      	|
|  [LightningModule]NeRFSystem.on_before_optimizer_step                                                                                                                                                       	|  2.1869e-06     	|  20000          	|  0.043737       	|  0.0014037      	|
|  [LightningModule]NeRFSystem.on_before_backward                                                                                                                                                             	|  2.1639e-06     	|  20000          	|  0.043278       	|  0.001389       	|
|  [LightningModule]NeRFSystem.validation_epoch_end                                                                                                                                                           	|  0.0068153      	|  2              	|  0.013631       	|  0.00043746     	|
|  [LightningModule]NeRFSystem.configure_optimizers                                                                                                                                                           	|  0.0083943      	|  1              	|  0.0083943      	|  0.00026941     	|
|  [Callback]TQDMProgressBar.on_train_epoch_start                                                                                                                                                             	|  0.00031308     	|  20             	|  0.0062616      	|  0.00020096     	|
|  [Callback]TQDMProgressBar.on_validation_batch_start                                                                                                                                                        	|  2.1271e-05     	|  260            	|  0.0055304      	|  0.00017749     	|
|  [Callback]TQDMProgressBar.on_train_epoch_end                                                                                                                                                               	|  0.00024532     	|  20             	|  0.0049065      	|  0.00015747     	|
|  [LightningModule]NeRFSystem.lr_scheduler_step                                                                                                                                                              	|  0.00011314     	|  20             	|  0.0022627      	|  7.2621e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_end                                                                                                                                            	|  7.9118e-06     	|  260            	|  0.0020571      	|  6.602e-05      	|
|  [Callback]TQDMProgressBar.on_validation_end                                                                                                                                                                	|  0.0007856      	|  2              	|  0.0015712      	|  5.0426e-05     	|
|  [LightningModule]NeRFSystem.validation_step_end                                                                                                                                                            	|  4.9968e-06     	|  260            	|  0.0012992      	|  4.1696e-05     	|
|  [Callback]GradientAccumulationScheduler.on_validation_batch_start                                                                                                                                          	|  4.8536e-06     	|  260            	|  0.0012619      	|  4.0501e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_end            	|  4.4883e-06     	|  260            	|  0.001167       	|  3.7453e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_eval                                                                                                                                                       	|  0.00055638     	|  2              	|  0.0011128      	|  3.5713e-05     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_batch_start          	|  4.2369e-06     	|  260            	|  0.0011016      	|  3.5355e-05     	|
|  [LightningModule]NeRFSystem.on_validation_model_train                                                                                                                                                      	|  0.00052181     	|  2              	|  0.0010436      	|  3.3494e-05     	|
|  [Callback]TQDMProgressBar.on_validation_start                                                                                                                                                              	|  0.00045654     	|  2              	|  0.00091309     	|  2.9305e-05     	|
|  [LightningModule]NeRFSystem.train_dataloader                                                                                                                                                               	|  0.00082788     	|  1              	|  0.00082788     	|  2.657e-05      	|
|  [LightningModule]NeRFSystem.on_validation_batch_end                                                                                                                                                        	|  2.9392e-06     	|  260            	|  0.00076419     	|  2.4526e-05     	|
|  [Strategy]SingleDeviceStrategy.validation_step_end                                                                                                                                                         	|  2.6596e-06     	|  260            	|  0.00069149     	|  2.2193e-05     	|
|  [LightningModule]NeRFSystem.on_validation_batch_start                                                                                                                                                      	|  2.5892e-06     	|  260            	|  0.00067319     	|  2.1606e-05     	|
|  [Callback]TQDMProgressBar.on_train_start                                                                                                                                                                   	|  0.00041925     	|  1              	|  0.00041925     	|  1.3455e-05     	|
|  [LightningModule]NeRFSystem.val_dataloader                                                                                                                                                                 	|  0.0003383      	|  1              	|  0.0003383      	|  1.0858e-05     	|
|  [Callback]TQDMProgressBar.on_train_end                                                                                                                                                                     	|  0.00023655     	|  1              	|  0.00023655     	|  7.5918e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_start                                                                                                                                               	|  9.6556e-06     	|  20             	|  0.00019311     	|  6.1978e-06     	|
|  [Callback]GradientAccumulationScheduler.on_train_epoch_end                                                                                                                                                 	|  7.0641e-06     	|  20             	|  0.00014128     	|  4.5344e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_start                                                                                                                                                                   	|  5.2951e-06     	|  22             	|  0.00011649     	|  3.7387e-06     	|
|  [Callback]TQDMProgressBar.on_epoch_end                                                                                                                                                                     	|  4.4792e-06     	|  22             	|  9.8542e-05     	|  3.1626e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_end                                                                                                                                                       	|  3.984e-06      	|  22             	|  8.7648e-05     	|  2.813e-06      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_end                       	|  3.9745e-06     	|  22             	|  8.744e-05      	|  2.8063e-06     	|
|  [Callback]GradientAccumulationScheduler.on_epoch_start                                                                                                                                                     	|  3.9173e-06     	|  22             	|  8.618e-05      	|  2.7659e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_epoch_start                     	|  3.8952e-06     	|  22             	|  8.5694e-05     	|  2.7503e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_epoch_start               	|  4.2777e-06     	|  20             	|  8.5554e-05     	|  2.7458e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_end                  	|  3.9704e-05     	|  2              	|  7.9407e-05     	|  2.5485e-06     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.setup                              	|  6.0062e-05     	|  1              	|  6.0062e-05     	|  1.9276e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_end                                                                                                                                                                   	|  2.6e-06        	|  22             	|  5.7199e-05     	|  1.8358e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_end                                                                                                                                                             	|  2.6085e-06     	|  20             	|  5.217e-05      	|  1.6744e-06     	|
|  [LightningModule]NeRFSystem.on_epoch_start                                                                                                                                                                 	|  2.2285e-06     	|  22             	|  4.9027e-05     	|  1.5735e-06     	|
|  [LightningModule]NeRFSystem.on_validation_end                                                                                                                                                              	|  2.4339e-05     	|  2              	|  4.8677e-05     	|  1.5623e-06     	|
|  [LightningModule]NeRFSystem.on_train_epoch_start                                                                                                                                                           	|  2.3291e-06     	|  20             	|  4.6581e-05     	|  1.495e-06      	|
|  [LightningModule]NeRFSystem.configure_sharded_model                                                                                                                                                        	|  2.4584e-05     	|  1              	|  2.4584e-05     	|  7.8901e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_end                                                                                                                                                          	|  9.1838e-06     	|  2              	|  1.8368e-05     	|  5.8949e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_end                                                                                                                                                  	|  8.0667e-06     	|  2              	|  1.6133e-05     	|  5.1779e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_start                                                                                                                                                	|  7.8217e-06     	|  2              	|  1.5643e-05     	|  5.0206e-07     	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_end                                                                                                                                            	|  6.8797e-06     	|  2              	|  1.3759e-05     	|  4.416e-07      	|
|  [Callback]TQDMProgressBar.on_fit_end                                                                                                                                                                       	|  1.3269e-05     	|  1              	|  1.3269e-05     	|  4.2584e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_start                                                                                                                                                        	|  1.2711e-05     	|  1              	|  1.2711e-05     	|  4.0794e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_end            	|  5.6224e-06     	|  2              	|  1.1245e-05     	|  3.6089e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_start                     	|  1.0476e-05     	|  1              	|  1.0476e-05     	|  3.362e-07      	|
|  [LightningModule]NeRFSystem.configure_callbacks                                                                                                                                                            	|  1.0406e-05     	|  1              	|  1.0406e-05     	|  3.3396e-07     	|
|  [LightningModule]NeRFSystem.on_fit_start                                                                                                                                                                   	|  9.4976e-06     	|  1              	|  9.4976e-06     	|  3.0482e-07     	|
|  [Callback]TQDMProgressBar.setup                                                                                                                                                                            	|  9.4278e-06     	|  1              	|  9.4278e-06     	|  3.0258e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_start                                                                                                                                                     	|  8.4508e-06     	|  1              	|  8.4508e-06     	|  2.7122e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_start                                                                                                                                                         	|  4.0163e-06     	|  2              	|  8.0327e-06     	|  2.578e-07      	|
|  [Callback]GradientAccumulationScheduler.on_validation_epoch_start                                                                                                                                          	|  4.0163e-06     	|  2              	|  8.0327e-06     	|  2.578e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_start                	|  4.0159e-06     	|  2              	|  8.0317e-06     	|  2.5777e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_end                                                                                                                                                        	|  3.8412e-06     	|  2              	|  7.6825e-06     	|  2.4656e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_validation_epoch_start          	|  3.8068e-06     	|  2              	|  7.6136e-06     	|  2.4435e-07     	|
|  [Callback]TQDMProgressBar.teardown                                                                                                                                                                         	|  7.6117e-06     	|  1              	|  7.6117e-06     	|  2.4429e-07     	|
|  [Callback]GradientAccumulationScheduler.teardown                                                                                                                                                           	|  7.6117e-06     	|  1              	|  7.6117e-06     	|  2.4429e-07     	|
|  [Strategy]SingleDeviceStrategy.on_validation_end                                                                                                                                                           	|  3.7714e-06     	|  2              	|  7.5428e-06     	|  2.4208e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.teardown                           	|  7.5428e-06     	|  1              	|  7.5428e-06     	|  2.4208e-07     	|
|  [Callback]TQDMProgressBar.on_validation_epoch_start                                                                                                                                                        	|  3.7709e-06     	|  2              	|  7.5419e-06     	|  2.4205e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_end                                                                                                                                                         	|  7.1237e-06     	|  1              	|  7.1237e-06     	|  2.2863e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_end                         	|  7.1228e-06     	|  1              	|  7.1228e-06     	|  2.286e-07      	|
|  [LightningModule]NeRFSystem.on_train_end                                                                                                                                                                   	|  5.7276e-06     	|  1              	|  5.7276e-06     	|  1.8382e-07     	|
|  [Callback]GradientAccumulationScheduler.on_fit_start                                                                                                                                                       	|  5.7258e-06     	|  1              	|  5.7258e-06     	|  1.8376e-07     	|
|  [Callback]GradientAccumulationScheduler.on_train_end                                                                                                                                                       	|  5.6578e-06     	|  1              	|  5.6578e-06     	|  1.8158e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_before_accelerator_backend_setup	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  1.8155e-07     	|
|  [Callback]GradientAccumulationScheduler.setup                                                                                                                                                              	|  5.6569e-06     	|  1              	|  5.6569e-06     	|  1.8155e-07     	|
|  [Callback]TQDMProgressBar.on_configure_sharded_model                                                                                                                                                       	|  5.587e-06      	|  1              	|  5.587e-06      	|  1.7931e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_configure_sharded_model         	|  5.2387e-06     	|  1              	|  5.2387e-06     	|  1.6813e-07     	|
|  [LightningModule]NeRFSystem.prepare_data                                                                                                                                                                   	|  5.2378e-06     	|  1              	|  5.2378e-06     	|  1.681e-07      	|
|  [Callback]TQDMProgressBar.on_before_accelerator_backend_setup                                                                                                                                              	|  5.1679e-06     	|  1              	|  5.1679e-06     	|  1.6586e-07     	|
|  [Callback]TQDMProgressBar.on_fit_start                                                                                                                                                                     	|  4.7497e-06     	|  1              	|  4.7497e-06     	|  1.5244e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_fit_start                       	|  4.679e-06      	|  1              	|  4.679e-06      	|  1.5017e-07     	|
|  [LightningModule]NeRFSystem.on_validation_epoch_start                                                                                                                                                      	|  2.1304e-06     	|  2              	|  4.2608e-06     	|  1.3675e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_start                                                                                                                                          	|  3.9116e-06     	|  1              	|  3.9116e-06     	|  1.2554e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_train_end                       	|  3.9106e-06     	|  1              	|  3.9106e-06     	|  1.2551e-07     	|
|  [Callback]GradientAccumulationScheduler.on_pretrain_routine_end                                                                                                                                            	|  3.8417e-06     	|  1              	|  3.8417e-06     	|  1.233e-07      	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_start          	|  3.8408e-06     	|  1              	|  3.8408e-06     	|  1.2327e-07     	|
|  [LightningModule]NeRFSystem.on_save_checkpoint                                                                                                                                                             	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.2105e-07     	|
|  [LightningModule]NeRFSystem.teardown                                                                                                                                                                       	|  3.7719e-06     	|  1              	|  3.7719e-06     	|  1.2105e-07     	|
|  [Callback]GradientAccumulationScheduler.on_configure_sharded_model                                                                                                                                         	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.2102e-07     	|
|  [Callback]TQDMProgressBar.on_pretrain_routine_end                                                                                                                                                          	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.2102e-07     	|
|  [Callback]ModelCheckpoint{'monitor': None, 'mode': 'min', 'every_n_train_steps': 0, 'every_n_epochs': 20, 'train_time_interval': None, 'save_on_train_epoch_end': True}.on_pretrain_routine_end            	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.2102e-07     	|
|  [LightningModule]NeRFSystem.on_fit_end                                                                                                                                                                     	|  3.7709e-06     	|  1              	|  3.7709e-06     	|  1.2102e-07     	|
|  [Callback]GradientAccumulationScheduler.on_before_accelerator_backend_setup                                                                                                                                	|  3.7011e-06     	|  1              	|  3.7011e-06     	|  1.1878e-07     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_start                                                                                                                                                      	|  3.282e-06      	|  1              	|  3.282e-06      	|  1.0533e-07     	|
|  [Strategy]SingleDeviceStrategy.on_train_start                                                                                                                                                              	|  2.8638e-06     	|  1              	|  2.8638e-06     	|  9.1912e-08     	|
|  [LightningModule]NeRFSystem.on_pretrain_routine_end                                                                                                                                                        	|  2.305e-06      	|  1              	|  2.305e-06      	|  7.3978e-08     	|
|  [Strategy]SingleDeviceStrategy.on_train_end                                                                                                                                                                	|  1.9548e-06     	|  1              	|  1.9548e-06     	|  6.2739e-08     	|
|  [LightningModule]NeRFSystem.on_train_start                                                                                                                                                                 	|  1.8161e-06     	|  1              	|  1.8161e-06     	|  5.8286e-08     	|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
