Skip to main content
ENHU
Home

Main navigation

  • Discover
    • News
    • Events
    • Tenders
  • Research fields
  • Resources
    • Publications
    • Downloads
    • Brochure
  • About us
  • Partners
  1. Home
  2. Publications
The 9th International Conference on Learning Representations (ICLR)

Fooling a Complete Neural Network Verifier

Openreview.net
Széchenyi Plusz RRF
Abstract

The efficient and accurate characterization of the robustness of neural networks to input perturbation is an important open problem. Many approaches exist including heuristic and exact (or complete) methods. Complete methods are expensive but their mathematical formulation guarantees that they provide exact robustness metrics. However, this guarantee is valid only if we assume that the verified network applies arbitrary-precision arithmetic and the verifier is reliable. In practice, however, both the networks and the verifiers apply limited-precision floating point arithmetic. In this paper, we show that numerical roundoff errors can be exploited to craft adversarial networks, in which the actual robustness and the robustness computed by a state-of-the-art complete verifier radically differ. We also show that such adversarial networks can be used to insert a backdoor into any network in such a way that the backdoor is completely missed by the verifier. The attack is easy to detect in its naive form but, as we show, the adversarial network can be transformed to make its detection less trivial. We offer a simple defense against our particular attack based on adding a very small perturbation to the network weights. However, our conjecture is that other numerical attacks are possible, and exact verification has to take into account all the details of the computation executed by the verified networks, which makes the problem significantly harder

Authors
Dániel Zombori
Balázs Bánhelyi
Tibor Csendes
István Megyeri
Márk Jelasity
Institutes
Read more
Home

LinkedIn

Become a partner

Subscribe to newsletter

Send partnership request

Explore

  • News
  • Events
  • Tenders
  • Publications
  • Downloads
  • Partners

Research fields

  • Foundations of AI
  • Human Language Processing
  • Machine perception
  • Medical, Health and Biology
  • Security and Privacy
  • Sensors, IoT and Telecommunications

Contact us

Hungary, H-1111 Budapest,
Kende u. 13-17.
+36 1 279 6000
@email

© 2020-2021 Artifical Intelligence National Laboratory, Budapest