BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//UC Irvine//CML Seminars//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Understanding the Diffusion Objective
X-WR-TIMEZONE:America/Los_Angeles
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20070311T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20071104T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:2023-04-10-durk-kingma@cml.ics.uci.edu
DTSTAMP:20230410T000000Z
SEQUENCE:57690
DTSTART;TZID=America/Los_Angeles:20230410T130000
DTEND;TZID=America/Los_Angeles:20230410T140000
SUMMARY:[CML Seminar] Durk Kingma: Understanding the Diffusion Objective
LOCATION:Donald Bren Hall 4011
DESCRIPTION:Durk Kingma\, Research Scientist\, Google Research\n\nTitle: Un
derstanding the Diffusion Objective\n\nAbstract: Some believe that maximum
likelihood is incompatible with high-quality image generation. We provide
counter-evidence: diffusion models with SOTA FIDs are actually optimized
with the ELBO\, with very simple data augmentation (additive noise). We sh
ow that diffusion models in the literature are optimized with various obje
ctives that are special cases of a weighted loss\, where the weighting fun
ction specifies the weight per noise level. Uniform weighting corresponds
to maximizing the ELBO\, a principled approximation of maximum likelihood.
We expose a direct relationship between the weighted loss (with any weigh
ting) and the ELBO objective: the weighted loss can be written as a weight
ed integral of ELBOs\, with one ELBO per noise level. If the weighting fun
ction is monotonic\, then the weighted loss is a likelihood-based objectiv
e. Our main contribution is a deeper theoretical understanding of the diff
usion objective.\n\nhttps://cml.ics.uci.edu/seminars/2023-04-10-durk-kingm
a
X-ALT-DESC;FMTTYPE=text/html:Durk Kingma\, Research Scie
ntist\, Google Research
Title: Understanding the Diffusion O
bjective
Abstract: Some believe that maximum likelihood is i
ncompatible with high-quality image generation. We provide counter-evidenc
e: diffusion models with SOTA FIDs are actually optimized with the ELBO\,
with very simple data augmentation (additive noise). We show that diffusio
n models in the literature are optimized with various objectives that are
special cases of a weighted loss\, where the weighting function specifies
the weight per noise level. Uniform weighting corresponds to maximizing th
e ELBO\, a principled approximation of maximum likelihood. We expose a dir
ect relationship between the weighted loss (with any weighting) and the EL
BO objective: the weighted loss can be written as a weighted integral of E
LBOs\, with one ELBO per noise level. If the weighting function is monoton
ic\, then the weighted loss is a likelihood-based objective. Our main cont
ribution is a deeper theoretical understanding of the diffusion objective.
https://cml.ics.uci.edu/seminars/2023-04-10-durk-kingma
URL:https://cml.ics.uci.edu/seminars/2023-04-10-durk-kingma
END:VEVENT
END:VCALENDAR